[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:04 dist-test-slave-7z32 krb5kdc[8399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:04 dist-test-slave-7z32 krb5kdc[8399](info): set up 2 sockets
Aug 06 16:06:04 dist-test-slave-7z32 krb5kdc[8399](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 16:06:06.251188 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.341s user 0.007s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:06 dist-test-slave-7z32 krb5kdc[8399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:06:06.436758 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:36389
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:36389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:06.579414 8421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:06.579830 8421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:06.579877 8421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:06.585114 8421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:06.585227 8421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:06.585250 8421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:06.585270 8421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:06.585289 8421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:06.592083 8421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:36389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:36389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:06.593544 8421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:06.594930 8421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:06.607234 8427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:06.607234 8429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:06.607407 8421 server_base.cc:1034] running on GCE node
W20260806 16:06:06.608117 8426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:06.609048 8421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:06.611192 8421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:06.612731 8421 hybrid_clock.cc:648] HybridClock initialized: now 1786032366612661 us; error 84 us; skew 500 ppm
Aug 06 16:06:06 dist-test-slave-7z32 krb5kdc[8399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:06.616847 8421 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:06:06.618223 8421 webserver.cc:460] Webserver started at http://127.8.49.62:36145/ using document root <none> and password file <none>
I20260806 16:06:06.619108 8421 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:06.619194 8421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:06.619445 8421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:06.622251 8421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "8cb88106dcb148048cf4057b7da22f6e"
format_stamp: "Formatted at 2026-08-06 16:06:06 on dist-test-slave-7z32"
server_key: "0369af56b06c27eff54dd4acbd0a9f3b"
server_key_iv: "360970fe680dfd5a6e75b3df89727c0c"
server_key_version: "encryptionkey@0"
I20260806 16:06:06.623191 8421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "8cb88106dcb148048cf4057b7da22f6e"
format_stamp: "Formatted at 2026-08-06 16:06:06 on dist-test-slave-7z32"
server_key: "0369af56b06c27eff54dd4acbd0a9f3b"
server_key_iv: "360970fe680dfd5a6e75b3df89727c0c"
server_key_version: "encryptionkey@0"
I20260806 16:06:06.629283 8421 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 16:06:06.633903 8441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:06.635999 8421 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:06.636226 8421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "8cb88106dcb148048cf4057b7da22f6e"
format_stamp: "Formatted at 2026-08-06 16:06:06 on dist-test-slave-7z32"
server_key: "0369af56b06c27eff54dd4acbd0a9f3b"
server_key_iv: "360970fe680dfd5a6e75b3df89727c0c"
server_key_version: "encryptionkey@0"
I20260806 16:06:06.636456 8421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:06.697727 8421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:06.702793 8421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:06.703079 8421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:06.715986 8421 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:36389
I20260806 16:06:06.716008 8516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:36389 every 8 connection(s)
I20260806 16:06:06.717474 8421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:06.719877 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 8421
I20260806 16:06:06.720098 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:06.721998 8388 external_mini_cluster.cc:1428] Setting key 2943857c9a460dc5df67fe869720b511
I20260806 16:06:06.722080 8518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:06.730561 8518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e: Bootstrap starting.
I20260806 16:06:06.733517 8518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:06.734472 8518 log.cc:824] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:06.736903 8518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e: No bootstrap required, opened a new log
I20260806 16:06:06.740937 8518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb88106dcb148048cf4057b7da22f6e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 36389 } }
I20260806 16:06:06.741304 8518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:06.741410 8518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb88106dcb148048cf4057b7da22f6e, State: Initialized, Role: FOLLOWER
I20260806 16:06:06.741973 8518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb88106dcb148048cf4057b7da22f6e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 36389 } }
I20260806 16:06:06.742161 8518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:06.742235 8518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:06.742348 8518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:06.743467 8518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb88106dcb148048cf4057b7da22f6e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 36389 } }
I20260806 16:06:06.743839 8518 leader_election.cc:302] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cb88106dcb148048cf4057b7da22f6e; no voters:
I20260806 16:06:06.744244 8518 leader_election.cc:288] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:06.744576 8527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:06.744982 8527 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [term 1 LEADER]: Becoming Leader. State: Replica: 8cb88106dcb148048cf4057b7da22f6e, State: Running, Role: LEADER
I20260806 16:06:06.745484 8527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb88106dcb148048cf4057b7da22f6e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 36389 } }
I20260806 16:06:06.745826 8518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 16:06:06 dist-test-slave-7z32 krb5kdc[8399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:06.747392 8529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cb88106dcb148048cf4057b7da22f6e. Latest consensus state: current_term: 1 leader_uuid: "8cb88106dcb148048cf4057b7da22f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb88106dcb148048cf4057b7da22f6e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 36389 } } }
I20260806 16:06:06.747551 8529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:06.749084 8540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:06.749516 8528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cb88106dcb148048cf4057b7da22f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb88106dcb148048cf4057b7da22f6e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 36389 } } }
I20260806 16:06:06.749678 8528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:06.754221 8540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:06.756570 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:06.727047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58294 (local address 127.8.49.62:36389)
0806 16:06:06.727784 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:06.727814 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:06.729077 (+ 1263us) server_negotiation.cc:407] Connection header received
0806 16:06:06.734091 (+ 5014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:06.734118 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:06.734667 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:06.735209 (+ 542us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:06.738745 (+ 3536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:06.740033 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:06.741684 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:06.742425 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:06.742626 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:06.748933 (+ 6307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:06.748945 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:06.748971 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:06.749044 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:06.751377 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:06.752321 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:06.752330 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:06.752431 (+ 101us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:06.752604 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:06.753116 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:06.753120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:06.753122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:06.753847 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:06.754060 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:06.755025 (+ 965us) server_negotiation.cc:299] Negotiation successful
0806 16:06:06.755427 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":245,"threads_started":1}
I20260806 16:06:06.762259 8540 catalog_manager.cc:1347] Generated new cluster ID: 96890ea39fa4412b8d9d8df46cb4593f
I20260806 16:06:06.762383 8540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:06.785168 8540 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:06.786547 8540 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:06.796242 8540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cb88106dcb148048cf4057b7da22f6e: Generated new TSK 0
I20260806 16:06:06.797359 8540 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:06:06.844725 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:36389
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:07.008651 8553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:07.009049 8553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:07.009104 8553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:07.013652 8553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:07.013751 8553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:07.013906 8553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:07.020632 8553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:36389
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:07.022269 8553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:07.023813 8553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:07.037235 8559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:07.037352 8560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:07.037556 8553 server_base.cc:1034] running on GCE node
W20260806 16:06:07.037243 8565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:07.038340 8553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:07.039500 8553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:07.040755 8553 hybrid_clock.cc:648] HybridClock initialized: now 1786032367040733 us; error 44 us; skew 500 ppm
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[8399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:07.044688 8553 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:06:07.045991 8553 webserver.cc:460] Webserver started at http://127.8.49.1:34047/ using document root <none> and password file <none>
I20260806 16:06:07.046792 8553 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:07.046919 8553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:07.047258 8553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:07.049805 8553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "cae2e316435a457c844aecec8c0dc561"
format_stamp: "Formatted at 2026-08-06 16:06:07 on dist-test-slave-7z32"
server_key: "763be735022a05b32178145651fe0283"
server_key_iv: "9f809dddb6c0fdd7ddc1d23eebd913f6"
server_key_version: "encryptionkey@0"
I20260806 16:06:07.050486 8553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "cae2e316435a457c844aecec8c0dc561"
format_stamp: "Formatted at 2026-08-06 16:06:07 on dist-test-slave-7z32"
server_key: "763be735022a05b32178145651fe0283"
server_key_iv: "9f809dddb6c0fdd7ddc1d23eebd913f6"
server_key_version: "encryptionkey@0"
I20260806 16:06:07.055649 8553 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:06:07.059682 8575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:07.061658 8553 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:06:07.061885 8553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "cae2e316435a457c844aecec8c0dc561"
format_stamp: "Formatted at 2026-08-06 16:06:07 on dist-test-slave-7z32"
server_key: "763be735022a05b32178145651fe0283"
server_key_iv: "9f809dddb6c0fdd7ddc1d23eebd913f6"
server_key_version: "encryptionkey@0"
I20260806 16:06:07.062062 8553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:07.080381 8553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:07.084512 8553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:07.084818 8553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:07.085608 8553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:07.086680 8553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:07.086759 8553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:07.086856 8553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:07.086902 8553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 16:06:07.102486 8553 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:37281
I20260806 16:06:07.102608 8730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:37281 every 8 connection(s)
I20260806 16:06:07.103729 8553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:07.108698 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 8553
I20260806 16:06:07.108820 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:07.109122 8388 external_mini_cluster.cc:1428] Setting key 5c11cd1f28002f990b523e7c7bd428a9
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[8399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:06:07.125453 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:07.106798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:33223 (local address 127.8.49.62:36389)
0806 16:06:07.107057 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:07.107064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:07.108403 (+ 1339us) server_negotiation.cc:407] Connection header received
0806 16:06:07.110567 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:07.110571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:07.110624 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:07.110740 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:07.112962 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.113851 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.115706 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.115984 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.116120 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:07.119757 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:07.119761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:07.119763 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:07.119797 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:07.121930 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:07.122765 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:07.122771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:07.122772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:07.122995 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:07.123549 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:07.123552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:07.123553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:07.123857 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:07.123971 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:07.124838 (+ 867us) server_negotiation.cc:299] Negotiation successful
0806 16:06:07.125173 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 16:06:07.126673 8734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:07.107288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:36389 (local address 127.8.49.1:33223)
0806 16:06:07.108074 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:07.108147 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:07.110235 (+ 2088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:07.111189 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:07.111201 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:07.111752 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:07.112490 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:07.112511 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.114071 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.114075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:07.115525 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:07.115542 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.116196 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.116202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:07.116449 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:07.117356 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:07.117379 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:07.119485 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:07.122112 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:07.122121 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:07.122152 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:07.122542 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:07.123161 (+ 619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:07.123165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:07.123167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:07.123418 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:07.124018 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:07.124028 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:07.124431 (+ 403us) client_negotiation.cc:769] Sending connection context
0806 16:06:07.124675 (+ 244us) client_negotiation.cc:240] Negotiation successful
0806 16:06:07.125188 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":140,"threads_started":1}
I20260806 16:06:07.128911 8731 heartbeater.cc:347] Connected to a master server at 127.8.49.62:36389
I20260806 16:06:07.129324 8731 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:07.130055 8731 heartbeater.cc:511] Master 127.8.49.62:36389 requested a full tablet report, sending...
I20260806 16:06:07.132412 8466 ts_manager.cc:194] Registered new tserver with Master: cae2e316435a457c844aecec8c0dc561 (127.8.49.1:37281)
I20260806 16:06:07.133800 8466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:33223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:06:07.151239 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:36389
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:07.316814 8738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:07.317339 8738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:07.317445 8738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:07.322270 8738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:07.322422 8738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:07.322536 8738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:07.329221 8738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:36389
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:07.330955 8738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:07.332538 8738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:07.346381 8749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:07.346380 8747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:07.346498 8746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:07.347115 8738 server_base.cc:1034] running on GCE node
I20260806 16:06:07.347786 8738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:07.348660 8738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:07.349962 8738 hybrid_clock.cc:648] HybridClock initialized: now 1786032367349900 us; error 86 us; skew 500 ppm
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[8399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:07.353524 8738 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:06:07.354765 8738 webserver.cc:460] Webserver started at http://127.8.49.2:43757/ using document root <none> and password file <none>
I20260806 16:06:07.355556 8738 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:07.355685 8738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:07.355952 8738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:07.358281 8738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "ae58e390fc3e44dc93e26562cb9ddd47"
format_stamp: "Formatted at 2026-08-06 16:06:07 on dist-test-slave-7z32"
server_key: "7796915fbe2d7966a2a85b806905903c"
server_key_iv: "72e0587f2dcd6802e411b7e955718b39"
server_key_version: "encryptionkey@0"
I20260806 16:06:07.359011 8738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "ae58e390fc3e44dc93e26562cb9ddd47"
format_stamp: "Formatted at 2026-08-06 16:06:07 on dist-test-slave-7z32"
server_key: "7796915fbe2d7966a2a85b806905903c"
server_key_iv: "72e0587f2dcd6802e411b7e955718b39"
server_key_version: "encryptionkey@0"
I20260806 16:06:07.364148 8738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:06:07.367903 8758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:07.369843 8738 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:07.370059 8738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "ae58e390fc3e44dc93e26562cb9ddd47"
format_stamp: "Formatted at 2026-08-06 16:06:07 on dist-test-slave-7z32"
server_key: "7796915fbe2d7966a2a85b806905903c"
server_key_iv: "72e0587f2dcd6802e411b7e955718b39"
server_key_version: "encryptionkey@0"
I20260806 16:06:07.370222 8738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:07.390182 8738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:07.394791 8738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:07.395115 8738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:07.395891 8738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:07.397190 8738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:07.397281 8738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:07.397365 8738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:07.397410 8738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:07.412355 8738 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:37139
I20260806 16:06:07.412456 8909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:37139 every 8 connection(s)
I20260806 16:06:07.413534 8738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:07.423203 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 8738
I20260806 16:06:07.423348 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:07.423621 8388 external_mini_cluster.cc:1428] Setting key 5dbcbb759407534c888271aa432fba16
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[8399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:07.435509 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:07.416426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:59491 (local address 127.8.49.62:36389)
0806 16:06:07.416633 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:07.416638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:07.417629 (+ 991us) server_negotiation.cc:407] Connection header received
0806 16:06:07.419517 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:07.419520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:07.419591 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:07.419695 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:07.421759 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.422366 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.424283 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.424622 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.424758 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:07.430008 (+ 5250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:07.430015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:07.430021 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:07.430104 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:07.432358 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:07.433377 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:07.433381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:07.433383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:07.433503 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:07.433953 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:07.433962 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:07.433963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:07.434219 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:07.434335 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:07.435094 (+ 759us) server_negotiation.cc:299] Negotiation successful
0806 16:06:07.435293 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 16:06:07.436928 8912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:07.416732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:36389 (local address 127.8.49.2:59491)
0806 16:06:07.417355 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:07.417424 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:07.419220 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:07.420168 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:07.420178 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:07.420775 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:07.421512 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:07.421531 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.422579 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.422583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:07.424082 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:07.424099 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.424895 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.424901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:07.425201 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:07.426514 (+ 1313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:07.426534 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:07.429530 (+ 2996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:07.432580 (+ 3050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:07.432591 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:07.432617 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:07.433171 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:07.433626 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:07.433631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:07.433633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:07.433834 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:07.434351 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:07.434363 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:07.434735 (+ 372us) client_negotiation.cc:769] Sending connection context
0806 16:06:07.435031 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 16:06:07.435526 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":142,"threads_started":1}
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:06:07.438735 8910 heartbeater.cc:347] Connected to a master server at 127.8.49.62:36389
I20260806 16:06:07.439160 8910 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:07.439862 8910 heartbeater.cc:511] Master 127.8.49.62:36389 requested a full tablet report, sending...
I20260806 16:06:07.441445 8466 ts_manager.cc:194] Registered new tserver with Master: ae58e390fc3e44dc93e26562cb9ddd47 (127.8.49.2:37139)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:06:07.442505 8466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:59491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:06:07.464022 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:36389
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:07.600868 8918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:07.601224 8918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:07.601275 8918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:07.605935 8918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:07.606021 8918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:07.606124 8918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:07.612222 8918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:36389
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:07.613584 8918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:07.615149 8918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:06:07.628508 8918 server_base.cc:1034] running on GCE node
W20260806 16:06:07.628440 8926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:07.628450 8924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:07.628440 8923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:07.629631 8918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:07.630457 8918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:07.631793 8918 hybrid_clock.cc:648] HybridClock initialized: now 1786032367631785 us; error 92 us; skew 500 ppm
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[8399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:07.643034 8918 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:06:07.644548 8918 webserver.cc:460] Webserver started at http://127.8.49.3:45199/ using document root <none> and password file <none>
I20260806 16:06:07.645255 8918 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:07.645330 8918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:07.645583 8918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:07.648243 8918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "9e3d6761703d40c7ad11bfc2713472a6"
format_stamp: "Formatted at 2026-08-06 16:06:07 on dist-test-slave-7z32"
server_key: "9a767f4688e4c0a73d85156c85f804c1"
server_key_iv: "970a731f8d00d9820f8d55e2d20320d6"
server_key_version: "encryptionkey@0"
I20260806 16:06:07.648936 8918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "9e3d6761703d40c7ad11bfc2713472a6"
format_stamp: "Formatted at 2026-08-06 16:06:07 on dist-test-slave-7z32"
server_key: "9a767f4688e4c0a73d85156c85f804c1"
server_key_iv: "970a731f8d00d9820f8d55e2d20320d6"
server_key_version: "encryptionkey@0"
I20260806 16:06:07.654775 8918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:06:07.658502 8935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:07.660331 8918 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:07.660550 8918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "9e3d6761703d40c7ad11bfc2713472a6"
format_stamp: "Formatted at 2026-08-06 16:06:07 on dist-test-slave-7z32"
server_key: "9a767f4688e4c0a73d85156c85f804c1"
server_key_iv: "970a731f8d00d9820f8d55e2d20320d6"
server_key_version: "encryptionkey@0"
I20260806 16:06:07.660705 8918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:07.677794 8918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:07.681668 8918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:07.681914 8918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:07.682745 8918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:07.684084 8918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:07.684178 8918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:07.684262 8918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:07.684307 8918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:07.699682 8918 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:40565
I20260806 16:06:07.699702 9093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:40565 every 8 connection(s)
I20260806 16:06:07.701052 8918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:07.705224 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 8918
I20260806 16:06:07.705411 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:07.705731 8388 external_mini_cluster.cc:1428] Setting key b05c556ca2ceea8d17af3f46afd22eeb
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[8399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:07.724076 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:07.704249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:59491 (local address 127.8.49.62:36389)
0806 16:06:07.704680 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:07.704686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:07.706266 (+ 1580us) server_negotiation.cc:407] Connection header received
0806 16:06:07.709209 (+ 2943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:07.709213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:07.709278 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:07.709389 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:07.711806 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.712543 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.714256 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.714576 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.714723 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:07.718886 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:07.718891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:07.718899 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:07.718937 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:07.720783 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:07.721516 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:07.721520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:07.721521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:07.721604 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:07.722358 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:07.722362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:07.722364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:07.722663 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:07.722786 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:07.723705 (+ 919us) server_negotiation.cc:299] Negotiation successful
0806 16:06:07.723879 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 16:06:07.725323 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:07.704810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:36389 (local address 127.8.49.3:59491)
0806 16:06:07.705961 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:07.706043 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:07.708757 (+ 2714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:07.709697 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:07.709712 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:07.710555 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:07.711531 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:07.711558 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.712790 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.712794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:07.714013 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:07.714034 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.714949 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.714955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:07.715247 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:07.716282 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:07.716303 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:07.718523 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:07.720941 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:07.720949 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:07.720974 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:07.721406 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:07.721945 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:07.721948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:07.721951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:07.722242 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:07.722903 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:07.722912 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:07.723390 (+ 478us) client_negotiation.cc:769] Sending connection context
0806 16:06:07.723653 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 16:06:07.724133 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":214,"threads_started":1}
I20260806 16:06:07.727092 9094 heartbeater.cc:347] Connected to a master server at 127.8.49.62:36389
I20260806 16:06:07.727497 9094 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:07.728448 9094 heartbeater.cc:511] Master 127.8.49.62:36389 requested a full tablet report, sending...
I20260806 16:06:07.730343 8466 ts_manager.cc:194] Registered new tserver with Master: 9e3d6761703d40c7ad11bfc2713472a6 (127.8.49.3:40565)
I20260806 16:06:07.731128 8466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:59491
I20260806 16:06:07.736027 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[8399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032367, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[8399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032367, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:06:07.774619 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:07.760735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54498 (local address 127.8.49.1:37281)
0806 16:06:07.761223 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:07.761232 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:07.761268 (+ 36us) server_negotiation.cc:407] Connection header received
0806 16:06:07.761346 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:07.761354 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:07.761628 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:07.761812 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:07.762644 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.763767 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.764784 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.765314 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.765429 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:07.768888 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:07.768902 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:07.768923 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:07.768963 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:07.771572 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:07.772304 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:07.772311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:07.772312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:07.772448 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:07.772812 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:07.772815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:07.772816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:07.773172 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:07.773408 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:07.773850 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 16:06:07.774152 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":173,"threads_started":1}
W20260806 16:06:07.776321 8612 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54498
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[8399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[8399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:06:07.816674 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:07.804656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54506 (local address 127.8.49.1:37281)
0806 16:06:07.804943 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:07.804950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:07.805262 (+ 312us) server_negotiation.cc:407] Connection header received
0806 16:06:07.805351 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:07.805354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:07.805423 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:07.805541 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:07.806522 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.807310 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.808371 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:07.808686 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:07.808809 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:07.812012 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:07.812017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:07.812020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:07.812069 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:07.814013 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:07.814772 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:07.814776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:07.814777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:07.815033 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:07.815417 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:07.815420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:07.815422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:07.815674 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:07.815864 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:07.816184 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 16:06:07.816434 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260806 16:06:07.818440 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 8553
I20260806 16:06:07.838784 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 8738
I20260806 16:06:07.862694 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 8918
I20260806 16:06:07.876374 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 8421
2026-08-06T16:06:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3015 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[9111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[9111](info): set up 2 sockets
Aug 06 16:06:07 dist-test-slave-7z32 krb5kdc[9111](info): commencing operation
krb5kdc: starting...
W20260806 16:06:09.205346 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:09 dist-test-slave-7z32 krb5kdc[9111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:06:09.391515 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:43609
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:43609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:09.524176 9135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:09.524623 9135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:09.524677 9135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:09.529002 9135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:09.529093 9135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:09.529114 9135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:09.529132 9135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:09.529150 9135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:09.535082 9135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:43609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:43609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:09.536532 9135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:09.537810 9135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:09.548588 9145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:09.548658 9141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:09.548593 9142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:09.549085 9135 server_base.cc:1034] running on GCE node
I20260806 16:06:09.549743 9135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:09.551127 9135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:09.552362 9135 hybrid_clock.cc:648] HybridClock initialized: now 1786032369552325 us; error 61 us; skew 500 ppm
Aug 06 16:06:09 dist-test-slave-7z32 krb5kdc[9111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:09.555796 9135 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:06:09.557015 9135 webserver.cc:460] Webserver started at http://127.8.49.62:37099/ using document root <none> and password file <none>
I20260806 16:06:09.557629 9135 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:09.557699 9135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:09.557902 9135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:09.560351 9135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "a450f6723c77406a9443bf5c705edc2f"
format_stamp: "Formatted at 2026-08-06 16:06:09 on dist-test-slave-7z32"
server_key: "241f388e015e7a84addf9eb492a445b4"
server_key_iv: "5220ccc618e7aef83c08c68a5820ec72"
server_key_version: "encryptionkey@0"
I20260806 16:06:09.561013 9135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "a450f6723c77406a9443bf5c705edc2f"
format_stamp: "Formatted at 2026-08-06 16:06:09 on dist-test-slave-7z32"
server_key: "241f388e015e7a84addf9eb492a445b4"
server_key_iv: "5220ccc618e7aef83c08c68a5820ec72"
server_key_version: "encryptionkey@0"
I20260806 16:06:09.565918 9135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:06:09.569337 9156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:09.571278 9135 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:09.571472 9135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "a450f6723c77406a9443bf5c705edc2f"
format_stamp: "Formatted at 2026-08-06 16:06:09 on dist-test-slave-7z32"
server_key: "241f388e015e7a84addf9eb492a445b4"
server_key_iv: "5220ccc618e7aef83c08c68a5820ec72"
server_key_version: "encryptionkey@0"
I20260806 16:06:09.571611 9135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:09.620083 9135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:09.624086 9135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:09.624289 9135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:09.633358 9135 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:43609
I20260806 16:06:09.633493 9229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:43609 every 8 connection(s)
I20260806 16:06:09.634611 9135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:09.639102 9230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:09.643640 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 9135
I20260806 16:06:09.643776 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:09.644152 8388 external_mini_cluster.cc:1428] Setting key 0e3512a42b7450ae87f5b49eb88e6f9e
I20260806 16:06:09.648458 9230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f: Bootstrap starting.
I20260806 16:06:09.652299 9230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:09.653657 9230 log.cc:824] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:06:09 dist-test-slave-7z32 krb5kdc[9111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:09.657931 9230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f: No bootstrap required, opened a new log
I20260806 16:06:09.663038 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:09.646151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46702 (local address 127.8.49.62:43609)
0806 16:06:09.646769 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:09.646791 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:09.646921 (+ 130us) server_negotiation.cc:407] Connection header received
0806 16:06:09.647832 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:09.647855 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:09.648329 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:09.648801 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:09.649843 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:09.651115 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:09.652126 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:09.652789 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:09.652990 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:09.656953 (+ 3963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:09.656961 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:09.656986 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:09.657030 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:09.659460 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:09.660029 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:09.660036 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:09.660042 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:09.660188 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:09.660508 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:09.660512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:09.660514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:09.661030 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:09.661270 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:09.661594 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 16:06:09.661960 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":215,"threads_started":1}
I20260806 16:06:09.663069 9230 raft_consensus.cc:348] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a450f6723c77406a9443bf5c705edc2f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43609 } }
I20260806 16:06:09.663527 9230 raft_consensus.cc:374] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:09.663623 9230 raft_consensus.cc:729] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a450f6723c77406a9443bf5c705edc2f, State: Initialized, Role: FOLLOWER
I20260806 16:06:09.664227 9230 consensus_queue.cc:260] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a450f6723c77406a9443bf5c705edc2f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43609 } }
I20260806 16:06:09.664427 9230 raft_consensus.cc:388] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:09.664510 9230 raft_consensus.cc:482] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:09.664635 9230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:09.666188 9230 raft_consensus.cc:504] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a450f6723c77406a9443bf5c705edc2f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43609 } }
I20260806 16:06:09.666687 9230 leader_election.cc:302] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a450f6723c77406a9443bf5c705edc2f; no voters:
I20260806 16:06:09.667191 9230 leader_election.cc:288] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:09.667443 9235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:09.667877 9235 raft_consensus.cc:686] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [term 1 LEADER]: Becoming Leader. State: Replica: a450f6723c77406a9443bf5c705edc2f, State: Running, Role: LEADER
I20260806 16:06:09.668331 9235 consensus_queue.cc:237] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a450f6723c77406a9443bf5c705edc2f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43609 } }
I20260806 16:06:09.669121 9230 sys_catalog.cc:564] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:09.670979 9236 sys_catalog.cc:455] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a450f6723c77406a9443bf5c705edc2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a450f6723c77406a9443bf5c705edc2f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43609 } } }
I20260806 16:06:09.671173 9236 sys_catalog.cc:458] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:09.670912 9237 sys_catalog.cc:455] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a450f6723c77406a9443bf5c705edc2f. Latest consensus state: current_term: 1 leader_uuid: "a450f6723c77406a9443bf5c705edc2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a450f6723c77406a9443bf5c705edc2f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43609 } } }
I20260806 16:06:09.671547 9237 sys_catalog.cc:458] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f [sys.catalog]: This master's current role is: LEADER
W20260806 16:06:09.675953 9255 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:06:09.676127 9255 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:06:09.676231 9248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:09.677109 9248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:09.686570 9248 catalog_manager.cc:1347] Generated new cluster ID: efd770139fc549e3a5e6e12a21f4230d
I20260806 16:06:09.686697 9248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:09.698897 9248 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:09.700734 9248 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:09.713182 9248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a450f6723c77406a9443bf5c705edc2f: Generated new TSK 0
I20260806 16:06:09.714195 9248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:06:09.757532 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:43609
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:09.891026 9263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:09.891709 9263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:09.891806 9263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:09.896525 9263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:09.896648 9263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:09.896756 9263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:09.902926 9263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:43609
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:09.904281 9263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:09.905498 9263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:09.917541 9272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:09.917906 9268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:09.917989 9270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:09.918962 9263 server_base.cc:1034] running on GCE node
I20260806 16:06:09.919507 9263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:09.920244 9263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:09.921608 9263 hybrid_clock.cc:648] HybridClock initialized: now 1786032369921553 us; error 70 us; skew 500 ppm
Aug 06 16:06:09 dist-test-slave-7z32 krb5kdc[9111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:09.925894 9263 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:06:09.927297 9263 webserver.cc:460] Webserver started at http://127.8.49.1:38687/ using document root <none> and password file <none>
I20260806 16:06:09.928028 9263 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:09.928146 9263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:09.928470 9263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:09.931106 9263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "d81e1df7587b4821903920ce94c78f08"
format_stamp: "Formatted at 2026-08-06 16:06:09 on dist-test-slave-7z32"
server_key: "1ca1640dfbb5ae10729316360d119dfc"
server_key_iv: "6347bb03627de7e99b892154bec0d835"
server_key_version: "encryptionkey@0"
I20260806 16:06:09.931807 9263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "d81e1df7587b4821903920ce94c78f08"
format_stamp: "Formatted at 2026-08-06 16:06:09 on dist-test-slave-7z32"
server_key: "1ca1640dfbb5ae10729316360d119dfc"
server_key_iv: "6347bb03627de7e99b892154bec0d835"
server_key_version: "encryptionkey@0"
I20260806 16:06:09.936724 9263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:06:09.940209 9281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:09.942046 9263 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:09.942245 9263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "d81e1df7587b4821903920ce94c78f08"
format_stamp: "Formatted at 2026-08-06 16:06:09 on dist-test-slave-7z32"
server_key: "1ca1640dfbb5ae10729316360d119dfc"
server_key_iv: "6347bb03627de7e99b892154bec0d835"
server_key_version: "encryptionkey@0"
I20260806 16:06:09.942389 9263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:09.963828 9263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:09.968200 9263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:09.968463 9263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:09.969249 9263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:09.970405 9263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:09.970522 9263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:09.970615 9263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:09.970671 9263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:09.983685 9263 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:40435
I20260806 16:06:09.983925 9444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:40435 every 8 connection(s)
I20260806 16:06:09.985035 9263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:09.989908 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 9263
I20260806 16:06:09.990043 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:09.990509 8388 external_mini_cluster.cc:1428] Setting key 368b4e27d19f843a58b93c1c273bb7d6
Aug 06 16:06:09 dist-test-slave-7z32 krb5kdc[9111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
I20260806 16:06:10.009658 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:09.987802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:40063 (local address 127.8.49.62:43609)
0806 16:06:09.987962 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:09.987966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:09.989321 (+ 1355us) server_negotiation.cc:407] Connection header received
0806 16:06:09.991541 (+ 2220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:09.991546 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:09.991636 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:09.991771 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:09.994191 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:09.995434 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:09.997936 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:09.998411 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:09.998594 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.003383 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:10.003388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:10.003390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:10.003431 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:10.005899 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.007336 (+ 1437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.007341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.007343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.007469 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.007960 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.007963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.007965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.008230 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:10.008379 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:10.009200 (+ 821us) server_negotiation.cc:299] Negotiation successful
0806 16:06:10.009436 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 16:06:10.011118 9447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:09.988291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:43609 (local address 127.8.49.1:40063)
0806 16:06:09.989063 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:09.989135 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:09.991199 (+ 2064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:09.992092 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:09.992103 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:09.992846 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:09.993817 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:09.993848 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:09.995752 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:09.995761 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:09.997711 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:09.997738 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:09.998775 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:09.998787 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:09.999098 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.000453 (+ 1355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:10.000486 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:10.003076 (+ 2590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:10.006219 (+ 3143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:10.006368 (+ 149us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:10.006398 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:10.007159 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:10.007590 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:10.007594 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:10.007601 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:10.007832 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:10.008423 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:10.008434 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:10.008851 (+ 417us) client_negotiation.cc:769] Sending connection context
0806 16:06:10.009118 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 16:06:10.009641 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":193,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:06:10.013743 9445 heartbeater.cc:347] Connected to a master server at 127.8.49.62:43609
I20260806 16:06:10.014175 9445 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:10.015453 9445 heartbeater.cc:511] Master 127.8.49.62:43609 requested a full tablet report, sending...
I20260806 16:06:10.018293 9172 ts_manager.cc:194] Registered new tserver with Master: d81e1df7587b4821903920ce94c78f08 (127.8.49.1:40435)
I20260806 16:06:10.020009 9172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:40063
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:06:10.047958 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:43609
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:10.205686 9450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:10.206180 9450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:10.206250 9450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:10.212026 9450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:10.212217 9450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:10.212370 9450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:10.221836 9450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:43609
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:10.223600 9450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:10.225477 9450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:10.241170 9461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:10.241616 9450 server_base.cc:1034] running on GCE node
W20260806 16:06:10.241209 9460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:10.241209 9464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:10.242430 9450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:10.243330 9450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:10.244547 9450 hybrid_clock.cc:648] HybridClock initialized: now 1786032370244497 us; error 71 us; skew 500 ppm
Aug 06 16:06:10 dist-test-slave-7z32 krb5kdc[9111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:10.248330 9450 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:06:10.249514 9450 webserver.cc:460] Webserver started at http://127.8.49.2:46839/ using document root <none> and password file <none>
I20260806 16:06:10.250214 9450 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:10.250309 9450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:10.250574 9450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:10.253042 9450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "1187080107d24c79a842ea49a46571a1"
format_stamp: "Formatted at 2026-08-06 16:06:10 on dist-test-slave-7z32"
server_key: "288e48ed5e324c0e884cadb403de8e6b"
server_key_iv: "8af377ec4d3aca07485e1a8262bfce77"
server_key_version: "encryptionkey@0"
I20260806 16:06:10.253684 9450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "1187080107d24c79a842ea49a46571a1"
format_stamp: "Formatted at 2026-08-06 16:06:10 on dist-test-slave-7z32"
server_key: "288e48ed5e324c0e884cadb403de8e6b"
server_key_iv: "8af377ec4d3aca07485e1a8262bfce77"
server_key_version: "encryptionkey@0"
I20260806 16:06:10.258574 9450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 16:06:10.262216 9473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:10.264281 9450 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:10.264513 9450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "1187080107d24c79a842ea49a46571a1"
format_stamp: "Formatted at 2026-08-06 16:06:10 on dist-test-slave-7z32"
server_key: "288e48ed5e324c0e884cadb403de8e6b"
server_key_iv: "8af377ec4d3aca07485e1a8262bfce77"
server_key_version: "encryptionkey@0"
I20260806 16:06:10.264678 9450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:10.306010 9450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:10.310648 9450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:10.310943 9450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:10.311715 9450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:10.312808 9450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:10.312891 9450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:10.312973 9450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:10.313027 9450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:10.326297 9450 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:37997
I20260806 16:06:10.326354 9627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:37997 every 8 connection(s)
I20260806 16:06:10.327474 9450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:10.330556 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 9450
I20260806 16:06:10.330691 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:10.331038 8388 external_mini_cluster.cc:1428] Setting key 02a462c774186624a266879e29f4a441
Aug 06 16:06:10 dist-test-slave-7z32 krb5kdc[9111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:06:10.349586 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.330350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:53381 (local address 127.8.49.62:43609)
0806 16:06:10.330623 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:10.330629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:10.331896 (+ 1267us) server_negotiation.cc:407] Connection header received
0806 16:06:10.333826 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:10.333833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:10.333902 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:10.334012 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:10.336192 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.336743 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.338949 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.339292 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.339411 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.344088 (+ 4677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:10.344092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:10.344094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:10.344126 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:10.346047 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.346765 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.346769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.346771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.346931 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.347808 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.347812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.347814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.348067 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:10.348169 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:10.349126 (+ 957us) server_negotiation.cc:299] Negotiation successful
0806 16:06:10.349337 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260806 16:06:10.351123 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.330928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:43609 (local address 127.8.49.2:53381)
0806 16:06:10.331649 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:10.331731 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:10.333517 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:10.334245 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:10.334254 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:10.334999 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:10.335962 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.335992 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.336950 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.336957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.338719 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.338739 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.339562 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.339571 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.340085 (+ 514us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.340995 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:10.341030 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:10.343793 (+ 2763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:10.346213 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:10.346223 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:10.346248 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:10.346633 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:10.347309 (+ 676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:10.347314 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:10.347315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:10.347627 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:10.348295 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:10.348303 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:10.348791 (+ 488us) client_negotiation.cc:769] Sending connection context
0806 16:06:10.349019 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 16:06:10.349468 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":220,"threads_started":1}
I20260806 16:06:10.353080 9630 heartbeater.cc:347] Connected to a master server at 127.8.49.62:43609
I20260806 16:06:10.353458 9630 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:10.354211 9630 heartbeater.cc:511] Master 127.8.49.62:43609 requested a full tablet report, sending...
I20260806 16:06:10.355808 9172 ts_manager.cc:194] Registered new tserver with Master: 1187080107d24c79a842ea49a46571a1 (127.8.49.2:37997)
I20260806 16:06:10.356559 9172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:53381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:06:10.377290 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:43609
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:10.538270 9635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:10.538697 9635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:10.538867 9635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:10.544178 9635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:10.544303 9635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:10.544380 9635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:10.552027 9635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:43609
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:10.553793 9635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:10.555661 9635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:06:10.570091 9635 server_base.cc:1034] running on GCE node
W20260806 16:06:10.570048 9641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:10.570012 9640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:10.570362 9643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:10.571216 9635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:10.572212 9635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:10.573457 9635 hybrid_clock.cc:648] HybridClock initialized: now 1786032370573426 us; error 53 us; skew 500 ppm
Aug 06 16:06:10 dist-test-slave-7z32 krb5kdc[9111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:10.577734 9635 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:06:10.579613 9635 webserver.cc:460] Webserver started at http://127.8.49.3:42447/ using document root <none> and password file <none>
I20260806 16:06:10.580543 9635 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:10.580633 9635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:10.580940 9635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:10.584049 9635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "1d5009a09a2c4289ae9d0dd45e167821"
format_stamp: "Formatted at 2026-08-06 16:06:10 on dist-test-slave-7z32"
server_key: "d8c0a95c8a95c13ff7f4785e41ec65ae"
server_key_iv: "523cce5a60f4a31f6ea841ed71d542ee"
server_key_version: "encryptionkey@0"
I20260806 16:06:10.584867 9635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "1d5009a09a2c4289ae9d0dd45e167821"
format_stamp: "Formatted at 2026-08-06 16:06:10 on dist-test-slave-7z32"
server_key: "d8c0a95c8a95c13ff7f4785e41ec65ae"
server_key_iv: "523cce5a60f4a31f6ea841ed71d542ee"
server_key_version: "encryptionkey@0"
I20260806 16:06:10.591437 9635 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:06:10.595860 9651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:10.598021 9635 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:06:10.598295 9635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "1d5009a09a2c4289ae9d0dd45e167821"
format_stamp: "Formatted at 2026-08-06 16:06:10 on dist-test-slave-7z32"
server_key: "d8c0a95c8a95c13ff7f4785e41ec65ae"
server_key_iv: "523cce5a60f4a31f6ea841ed71d542ee"
server_key_version: "encryptionkey@0"
I20260806 16:06:10.598531 9635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:10.620743 9635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:10.625385 9635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:10.625690 9635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:10.626556 9635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:10.627764 9635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:10.627846 9635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:10.627928 9635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:10.627969 9635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:10.642393 9635 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:45363
I20260806 16:06:10.642473 9810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:45363 every 8 connection(s)
I20260806 16:06:10.643774 9635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:10.649772 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 9635
I20260806 16:06:10.649972 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:10.650365 8388 external_mini_cluster.cc:1428] Setting key f2ea8376a0bfeb15ddde52746bc64f84
Aug 06 16:06:10 dist-test-slave-7z32 krb5kdc[9111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:10.664501 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.646497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:59697 (local address 127.8.49.62:43609)
0806 16:06:10.647035 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:10.647044 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:10.648255 (+ 1211us) server_negotiation.cc:407] Connection header received
0806 16:06:10.650265 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:10.650271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:10.650368 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:10.650526 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:10.652744 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.653561 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.655135 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.655372 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.655477 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.659238 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:10.659243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:10.659245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:10.659285 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:10.661147 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.662110 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.662113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.662115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.662221 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.662764 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.662767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.662768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.663144 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:10.663248 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:10.664131 (+ 883us) server_negotiation.cc:299] Negotiation successful
0806 16:06:10.664279 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392}
I20260806 16:06:10.665827 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.647031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:43609 (local address 127.8.49.3:59697)
0806 16:06:10.647898 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:10.647964 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:10.649821 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:10.650947 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:10.650958 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:10.651671 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:10.652542 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.652567 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.653753 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.653757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.654983 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.654996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.655604 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.655608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.655891 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.656722 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:10.656742 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:10.658905 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:10.661360 (+ 2455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:10.661379 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:10.661401 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:10.661901 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:10.662365 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:10.662368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:10.662369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:10.662589 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:10.663288 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:10.663296 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:10.663748 (+ 452us) client_negotiation.cc:769] Sending connection context
0806 16:06:10.663998 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 16:06:10.664552 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":207,"threads_started":1}
I20260806 16:06:10.668128 9811 heartbeater.cc:347] Connected to a master server at 127.8.49.62:43609
I20260806 16:06:10.668702 9811 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:10.669986 9811 heartbeater.cc:511] Master 127.8.49.62:43609 requested a full tablet report, sending...
I20260806 16:06:10.671873 9172 ts_manager.cc:194] Registered new tserver with Master: 1d5009a09a2c4289ae9d0dd45e167821 (127.8.49.3:45363)
I20260806 16:06:10.672629 9172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:59697
I20260806 16:06:10.678498 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:06:10.692802 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.683179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46718 (local address 127.8.49.62:43609)
0806 16:06:10.683344 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:10.683348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:10.683771 (+ 423us) server_negotiation.cc:407] Connection header received
0806 16:06:10.684028 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:10.684032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:10.684091 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:10.684192 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:10.685056 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.685656 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.686590 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.686924 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.687051 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.688829 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:10.688834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:10.688847 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:10.688878 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:10.690659 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.691249 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.691253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.691255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.691355 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.691698 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.691702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.691704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.691991 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:10.692153 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:10.692465 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 16:06:10.692612 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:06:10.700821 9172 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:10.704857 9172 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:10.728468 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.719248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45930 (local address 127.8.49.1:40435)
0806 16:06:10.720062 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:10.720075 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:10.720730 (+ 655us) server_negotiation.cc:407] Connection header received
0806 16:06:10.721012 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:10.721019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:10.721318 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:10.721499 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:10.722108 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.723376 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.725666 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.726700 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.726881 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.727105 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:10.728061 (+ 956us) server_negotiation.cc:299] Negotiation successful
0806 16:06:10.728226 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":84,"threads_started":1}
I20260806 16:06:10.728514 9828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.720294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:37997 (local address 127.0.0.1:37698)
0806 16:06:10.721128 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:10.721146 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:10.721265 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:10.722019 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:10.722023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:10.722044 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:10.722164 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.722171 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.723750 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.723753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.725793 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.725800 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.727593 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.727600 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.727697 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.727796 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 16:06:10.727956 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 16:06:10.728268 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":444,"thread_start_us":102,"threads_started":1}
I20260806 16:06:10.728821 9824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.719357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:45363 (local address 127.0.0.1:56506)
0806 16:06:10.720499 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:10.720586 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:10.720783 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:10.721498 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:10.721506 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:10.721530 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:10.721734 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.721744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.723573 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.723578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.725779 (+ 2201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.725795 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.728174 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.728178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.728299 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.728360 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 16:06:10.728438 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 16:06:10.728583 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":324,"thread_start_us":269,"threads_started":1}
I20260806 16:06:10.728514 9826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.719090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40435 (local address 127.0.0.1:45930)
0806 16:06:10.720499 (+ 1409us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:10.720586 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:10.720799 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:10.721770 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:10.721774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:10.721798 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:10.721914 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.721920 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.723743 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.723748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.725444 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.725462 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.727024 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.727030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.727720 (+ 690us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.727786 (+ 66us) client_negotiation.cc:769] Sending connection context
0806 16:06:10.727963 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 16:06:10.728268 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086,"mutex_wait_us":383,"thread_start_us":146,"threads_started":1}
I20260806 16:06:10.729498 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.720698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37698 (local address 127.8.49.2:37997)
0806 16:06:10.721199 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:10.721212 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:10.721238 (+ 26us) server_negotiation.cc:407] Connection header received
0806 16:06:10.721440 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:10.721446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:10.721613 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:10.721780 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:10.722345 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.723583 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.726079 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.727002 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.727245 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.727486 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:10.729086 (+ 1600us) server_negotiation.cc:299] Negotiation successful
0806 16:06:10.729296 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":141,"threads_started":1}
I20260806 16:06:10.729924 9827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.719964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56506 (local address 127.8.49.3:45363)
0806 16:06:10.720855 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:10.720872 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:10.720906 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:06:10.721015 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:10.721023 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:10.721308 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:10.721504 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:10.721923 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.723381 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.725979 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.727975 (+ 1996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.729291 (+ 1316us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.729525 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:10.729620 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 16:06:10.729752 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":254,"threads_started":1}
I20260806 16:06:10.733677 9723 tablet_service.cc:1467] Processing CreateTablet for tablet 6b76e5cf87dd47f8a4ca44fd5e3cddac (DEFAULT_TABLE table=test-table [id=26bb34e526a046b3b3e5b85283db9df2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:10.734073 9352 tablet_service.cc:1467] Processing CreateTablet for tablet 6b76e5cf87dd47f8a4ca44fd5e3cddac (DEFAULT_TABLE table=test-table [id=26bb34e526a046b3b3e5b85283db9df2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:10.734072 9540 tablet_service.cc:1467] Processing CreateTablet for tablet 6b76e5cf87dd47f8a4ca44fd5e3cddac (DEFAULT_TABLE table=test-table [id=26bb34e526a046b3b3e5b85283db9df2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:10.735244 9723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b76e5cf87dd47f8a4ca44fd5e3cddac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:10.735648 9540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b76e5cf87dd47f8a4ca44fd5e3cddac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:10.735648 9352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b76e5cf87dd47f8a4ca44fd5e3cddac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:10.745234 9831 tablet_bootstrap.cc:492] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821: Bootstrap starting.
I20260806 16:06:10.746585 9833 tablet_bootstrap.cc:492] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08: Bootstrap starting.
I20260806 16:06:10.748064 9834 tablet_bootstrap.cc:492] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1: Bootstrap starting.
I20260806 16:06:10.748373 9831 tablet_bootstrap.cc:654] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:10.750348 9831 log.cc:824] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:10.751362 9833 tablet_bootstrap.cc:654] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:10.751922 9834 tablet_bootstrap.cc:654] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:10.752455 9833 log.cc:824] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:10.753033 9834 log.cc:824] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:10.753892 9831 tablet_bootstrap.cc:492] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821: No bootstrap required, opened a new log
I20260806 16:06:10.754170 9831 ts_tablet_manager.cc:1397] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260806 16:06:10.755270 9833 tablet_bootstrap.cc:492] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08: No bootstrap required, opened a new log
I20260806 16:06:10.755515 9833 ts_tablet_manager.cc:1397] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 16:06:10.757164 9834 tablet_bootstrap.cc:492] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1: No bootstrap required, opened a new log
I20260806 16:06:10.757473 9834 ts_tablet_manager.cc:1397] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260806 16:06:10.759174 9831 raft_consensus.cc:348] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d81e1df7587b4821903920ce94c78f08" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40435 } } peers { permanent_uuid: "1187080107d24c79a842ea49a46571a1" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37997 } } peers { permanent_uuid: "1d5009a09a2c4289ae9d0dd45e167821" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 45363 } }
I20260806 16:06:10.759573 9831 raft_consensus.cc:374] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:10.759645 9831 raft_consensus.cc:729] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5009a09a2c4289ae9d0dd45e167821, State: Initialized, Role: FOLLOWER
I20260806 16:06:10.760219 9831 consensus_queue.cc:260] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d81e1df7587b4821903920ce94c78f08" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40435 } } peers { permanent_uuid: "1187080107d24c79a842ea49a46571a1" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37997 } } peers { permanent_uuid: "1d5009a09a2c4289ae9d0dd45e167821" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 45363 } }
I20260806 16:06:10.761360 9811 heartbeater.cc:503] Master 127.8.49.62:43609 was elected leader, sending a full tablet report...
I20260806 16:06:10.761557 9831 ts_tablet_manager.cc:1428] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821: Time spent starting tablet: real 0.007s user 0.008s sys 0.001s
I20260806 16:06:10.762166 9833 raft_consensus.cc:348] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d81e1df7587b4821903920ce94c78f08" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40435 } } peers { permanent_uuid: "1187080107d24c79a842ea49a46571a1" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37997 } } peers { permanent_uuid: "1d5009a09a2c4289ae9d0dd45e167821" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 45363 } }
I20260806 16:06:10.762461 9833 raft_consensus.cc:374] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:10.762530 9833 raft_consensus.cc:729] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d81e1df7587b4821903920ce94c78f08, State: Initialized, Role: FOLLOWER
I20260806 16:06:10.762651 9834 raft_consensus.cc:348] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d81e1df7587b4821903920ce94c78f08" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40435 } } peers { permanent_uuid: "1187080107d24c79a842ea49a46571a1" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37997 } } peers { permanent_uuid: "1d5009a09a2c4289ae9d0dd45e167821" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 45363 } }
I20260806 16:06:10.763062 9834 raft_consensus.cc:374] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:10.763151 9834 raft_consensus.cc:729] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1187080107d24c79a842ea49a46571a1, State: Initialized, Role: FOLLOWER
I20260806 16:06:10.763078 9833 consensus_queue.cc:260] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d81e1df7587b4821903920ce94c78f08" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40435 } } peers { permanent_uuid: "1187080107d24c79a842ea49a46571a1" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37997 } } peers { permanent_uuid: "1d5009a09a2c4289ae9d0dd45e167821" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 45363 } }
I20260806 16:06:10.763738 9834 consensus_queue.cc:260] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d81e1df7587b4821903920ce94c78f08" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40435 } } peers { permanent_uuid: "1187080107d24c79a842ea49a46571a1" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37997 } } peers { permanent_uuid: "1d5009a09a2c4289ae9d0dd45e167821" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 45363 } }
I20260806 16:06:10.764192 9445 heartbeater.cc:503] Master 127.8.49.62:43609 was elected leader, sending a full tablet report...
I20260806 16:06:10.765076 9833 ts_tablet_manager.cc:1428] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260806 16:06:10.765295 9834 ts_tablet_manager.cc:1428] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260806 16:06:10.765745 9630 heartbeater.cc:503] Master 127.8.49.62:43609 was elected leader, sending a full tablet report...
W20260806 16:06:10.829358 9631 tablet.cc:2369] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:10.885093 9844 raft_consensus.cc:482] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:10.885416 9844 raft_consensus.cc:504] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d81e1df7587b4821903920ce94c78f08" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40435 } } peers { permanent_uuid: "1187080107d24c79a842ea49a46571a1" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37997 } } peers { permanent_uuid: "1d5009a09a2c4289ae9d0dd45e167821" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 45363 } }
I20260806 16:06:10.887302 9844 leader_election.cc:288] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d81e1df7587b4821903920ce94c78f08 (127.8.49.1:40435), 1d5009a09a2c4289ae9d0dd45e167821 (127.8.49.3:45363)
I20260806 16:06:10.892112 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.887721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:45363 (local address 127.8.49.2:54477)
0806 16:06:10.888130 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:10.888149 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:10.888404 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:10.888716 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:10.888719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:10.888745 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:10.888905 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.888912 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.889734 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.889738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.890752 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.890763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.891555 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.891559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.891679 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.891711 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 16:06:10.891806 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 16:06:10.891912 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":34,"thread_start_us":129,"threads_started":1}
I20260806 16:06:10.892091 9827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.887880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:54477 (local address 127.8.49.3:45363)
0806 16:06:10.888131 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:10.888138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:10.888322 (+ 184us) server_negotiation.cc:407] Connection header received
0806 16:06:10.888535 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:10.888539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:10.888595 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:10.888686 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:10.889029 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.889598 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.890944 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.891405 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.891537 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.891623 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:10.891828 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 16:06:10.891916 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260806 16:06:10.892624 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.887864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:34375 (local address 127.8.49.1:40435)
0806 16:06:10.888059 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:10.888065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:10.888558 (+ 493us) server_negotiation.cc:407] Connection header received
0806 16:06:10.888791 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:10.888794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:10.888853 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:10.888981 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:10.889328 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.889907 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.891303 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.891856 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.892010 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.892085 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:10.892364 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 16:06:10.892461 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:06:10.893116 9846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.887774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40435 (local address 127.8.49.2:34375)
0806 16:06:10.888349 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:10.888365 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:10.888639 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:10.889009 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:10.889012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:10.889033 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:10.889197 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.889204 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.890047 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.890052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.891156 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:10.891169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.892027 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.892031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:10.892146 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.892172 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:06:10.892845 (+ 673us) client_negotiation.cc:240] Negotiation successful
0806 16:06:10.892969 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":1,"thread_start_us":198,"threads_started":1}
I20260806 16:06:10.893115 9746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b76e5cf87dd47f8a4ca44fd5e3cddac" candidate_uuid: "1187080107d24c79a842ea49a46571a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5009a09a2c4289ae9d0dd45e167821" is_pre_election: true
I20260806 16:06:10.893594 9746 raft_consensus.cc:2454] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1187080107d24c79a842ea49a46571a1 in term 0.
I20260806 16:06:10.893997 9381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b76e5cf87dd47f8a4ca44fd5e3cddac" candidate_uuid: "1187080107d24c79a842ea49a46571a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d81e1df7587b4821903920ce94c78f08" is_pre_election: true
I20260806 16:06:10.894371 9381 raft_consensus.cc:2454] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1187080107d24c79a842ea49a46571a1 in term 0.
I20260806 16:06:10.894325 9476 leader_election.cc:302] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1187080107d24c79a842ea49a46571a1, 1d5009a09a2c4289ae9d0dd45e167821; no voters:
I20260806 16:06:10.894770 9844 raft_consensus.cc:2781] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:10.894917 9844 raft_consensus.cc:482] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:10.894984 9844 raft_consensus.cc:3037] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [term 0 FOLLOWER]: Advancing to term 1
W20260806 16:06:10.895102 9812 tablet.cc:2369] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:10.896503 9844 raft_consensus.cc:504] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d81e1df7587b4821903920ce94c78f08" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40435 } } peers { permanent_uuid: "1187080107d24c79a842ea49a46571a1" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37997 } } peers { permanent_uuid: "1d5009a09a2c4289ae9d0dd45e167821" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 45363 } }
I20260806 16:06:10.897033 9844 leader_election.cc:288] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [CANDIDATE]: Term 1 election: Requested vote from peers d81e1df7587b4821903920ce94c78f08 (127.8.49.1:40435), 1d5009a09a2c4289ae9d0dd45e167821 (127.8.49.3:45363)
I20260806 16:06:10.897722 9746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b76e5cf87dd47f8a4ca44fd5e3cddac" candidate_uuid: "1187080107d24c79a842ea49a46571a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5009a09a2c4289ae9d0dd45e167821"
I20260806 16:06:10.897876 9381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b76e5cf87dd47f8a4ca44fd5e3cddac" candidate_uuid: "1187080107d24c79a842ea49a46571a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d81e1df7587b4821903920ce94c78f08"
I20260806 16:06:10.897903 9746 raft_consensus.cc:3037] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:10.898101 9381 raft_consensus.cc:3037] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:10.899624 9381 raft_consensus.cc:2454] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1187080107d24c79a842ea49a46571a1 in term 1.
I20260806 16:06:10.899639 9746 raft_consensus.cc:2454] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1d5009a09a2c4289ae9d0dd45e167821 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1187080107d24c79a842ea49a46571a1 in term 1.
I20260806 16:06:10.900207 9474 leader_election.cc:302] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1187080107d24c79a842ea49a46571a1, d81e1df7587b4821903920ce94c78f08; no voters:
I20260806 16:06:10.900760 9844 raft_consensus.cc:2781] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:10.901221 9844 raft_consensus.cc:686] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [term 1 LEADER]: Becoming Leader. State: Replica: 1187080107d24c79a842ea49a46571a1, State: Running, Role: LEADER
I20260806 16:06:10.901749 9844 consensus_queue.cc:237] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d81e1df7587b4821903920ce94c78f08" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40435 } } peers { permanent_uuid: "1187080107d24c79a842ea49a46571a1" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37997 } } peers { permanent_uuid: "1d5009a09a2c4289ae9d0dd45e167821" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 45363 } }
I20260806 16:06:10.907042 9175 catalog_manager.cc:5495] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P 1187080107d24c79a842ea49a46571a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1187080107d24c79a842ea49a46571a1 (127.8.49.2). New cstate: current_term: 1 leader_uuid: "1187080107d24c79a842ea49a46571a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d81e1df7587b4821903920ce94c78f08" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1187080107d24c79a842ea49a46571a1" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d5009a09a2c4289ae9d0dd45e167821" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 45363 } health_report { overall_health: UNKNOWN } } }
Aug 06 16:06:10 dist-test-slave-7z32 krb5kdc[9111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:06:10.928726 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.915938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45946 (local address 127.8.49.1:40435)
0806 16:06:10.916213 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:10.916218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:10.916480 (+ 262us) server_negotiation.cc:407] Connection header received
0806 16:06:10.916624 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:10.916627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:10.916680 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:10.916784 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:10.917884 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.918467 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.919671 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.919972 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.920090 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.924391 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:10.924410 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:10.924434 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:10.924474 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:10.926491 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.927167 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.927173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.927174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.927265 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.927578 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.927581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.927583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.927898 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:10.928141 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:10.928389 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 16:06:10.928535 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
Aug 06 16:06:10 dist-test-slave-7z32 krb5kdc[9111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 16:06:10 dist-test-slave-7z32 krb5kdc[9111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:06:10.968304 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.957670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45960 (local address 127.8.49.1:40435)
0806 16:06:10.957835 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:10.957840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:10.958144 (+ 304us) server_negotiation.cc:407] Connection header received
0806 16:06:10.958431 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:10.958434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:10.958484 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:10.958584 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:10.959517 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.960049 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.960969 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.961204 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:10.961318 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:10.964425 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:10.964430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:10.964432 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:10.964470 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:10.966197 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.966747 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.966751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.966754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.966902 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:10.967234 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:10.967238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:10.967240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:10.967461 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:10.967615 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:10.967952 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 16:06:10.968099 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 16:06:10.987288 9446 tablet.cc:2369] T 6b76e5cf87dd47f8a4ca44fd5e3cddac P d81e1df7587b4821903920ce94c78f08: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 16:06:10 dist-test-slave-7z32 krb5kdc[9111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 16:06:11 dist-test-slave-7z32 krb5kdc[9111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:06:11.011214 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:10.996654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.8.49.1:40435)
0806 16:06:10.996807 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:10.996813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:10.997124 (+ 311us) server_negotiation.cc:407] Connection header received
0806 16:06:10.997356 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:10.997360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:10.997447 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:10.997574 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:10.999089 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:10.999895 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:11.002610 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:11.003099 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:11.003314 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:11.006632 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:11.006637 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:11.006640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:11.006690 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:11.008927 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:11.009539 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:11.009543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:11.009544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:11.009681 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:11.010025 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:11.010029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:11.010030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:11.010287 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:11.010411 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:11.010760 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 16:06:11.011010 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 16:06:11.013413 9325 tablet_service.cc:2908] Rejecting scan request for tablet 6b76e5cf87dd47f8a4ca44fd5e3cddac: Uninitialized: clean time has not yet been initialized
I20260806 16:06:11.017174 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 9263
I20260806 16:06:11.039599 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 9450
I20260806 16:06:11.056764 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 9635
I20260806 16:06:11.078308 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 9135
2026-08-06T16:06:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3207 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:11 dist-test-slave-7z32 krb5kdc[9866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:11 dist-test-slave-7z32 krb5kdc[9866](info): set up 2 sockets
Aug 06 16:06:11 dist-test-slave-7z32 krb5kdc[9866](info): commencing operation
krb5kdc: starting...
W20260806 16:06:12.427807 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.002s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:12 dist-test-slave-7z32 krb5kdc[9866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:06:12.602342 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:46497
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:46497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:12.737401 9885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:12.737918 9885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:12.738025 9885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:12.746110 9885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:12.746229 9885 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 16:06:12.746259 9885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:12.746299 9885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:12.746322 9885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:12.746347 9885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:12.757386 9885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:46497
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:46497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:12.759063 9885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:12.760550 9885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:12.773034 9892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:12.773063 9897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:12.773015 9895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:12.774582 9885 server_base.cc:1034] running on GCE node
I20260806 16:06:12.775434 9885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:12.776954 9885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:12.783026 9885 hybrid_clock.cc:648] HybridClock initialized: now 1786032372782958 us; error 82 us; skew 500 ppm
Aug 06 16:06:12 dist-test-slave-7z32 krb5kdc[9866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032372, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:12.789597 9885 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:06:12.791985 9885 webserver.cc:460] Webserver started at http://127.8.49.62:35393/ using document root <none> and password file <none>
I20260806 16:06:12.792997 9885 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:12.793090 9885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:12.793491 9885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:12.796901 9885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "69a14282e8c84da29c7865f0dc1cce06"
format_stamp: "Formatted at 2026-08-06 16:06:12 on dist-test-slave-7z32"
server_key: "cac434f3f6ad77d934c90352e9e41193"
server_key_iv: "ef2d4793673ec7e30bf24ea3c0b587bc"
server_key_version: "encryptionkey@0"
I20260806 16:06:12.797780 9885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "69a14282e8c84da29c7865f0dc1cce06"
format_stamp: "Formatted at 2026-08-06 16:06:12 on dist-test-slave-7z32"
server_key: "cac434f3f6ad77d934c90352e9e41193"
server_key_iv: "ef2d4793673ec7e30bf24ea3c0b587bc"
server_key_version: "encryptionkey@0"
I20260806 16:06:12.804060 9885 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:06:12.808148 9908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:12.810343 9885 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260806 16:06:12.810557 9885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "69a14282e8c84da29c7865f0dc1cce06"
format_stamp: "Formatted at 2026-08-06 16:06:12 on dist-test-slave-7z32"
server_key: "cac434f3f6ad77d934c90352e9e41193"
server_key_iv: "ef2d4793673ec7e30bf24ea3c0b587bc"
server_key_version: "encryptionkey@0"
I20260806 16:06:12.810729 9885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:12.836493 9885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:12.840799 9885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:12.841030 9885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:12.853600 9885 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:46497
I20260806 16:06:12.853662 10002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:46497 every 8 connection(s)
I20260806 16:06:12.855317 9885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:12.859278 10004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:12.863735 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 9885
I20260806 16:06:12.863907 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:12.864358 8388 external_mini_cluster.cc:1428] Setting key e0ee1ed9dc875df31ee32978c3ce3bb9
I20260806 16:06:12.865801 10004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06: Bootstrap starting.
I20260806 16:06:12.868548 10004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:12.869418 10004 log.cc:824] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:12.872233 10004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06: No bootstrap required, opened a new log
Aug 06 16:06:12 dist-test-slave-7z32 krb5kdc[9866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:12.877192 10004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a14282e8c84da29c7865f0dc1cce06" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46497 } }
I20260806 16:06:12.877483 10004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:12.877564 10004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a14282e8c84da29c7865f0dc1cce06, State: Initialized, Role: FOLLOWER
I20260806 16:06:12.878118 10004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a14282e8c84da29c7865f0dc1cce06" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46497 } }
I20260806 16:06:12.878270 10004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:12.878352 10004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:12.878494 10004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:12.879904 10004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a14282e8c84da29c7865f0dc1cce06" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46497 } }
I20260806 16:06:12.880337 10004 leader_election.cc:302] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69a14282e8c84da29c7865f0dc1cce06; no voters:
I20260806 16:06:12.880756 10004 leader_election.cc:288] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:12.880838 10009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:12.881129 10009 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [term 1 LEADER]: Becoming Leader. State: Replica: 69a14282e8c84da29c7865f0dc1cce06, State: Running, Role: LEADER
I20260806 16:06:12.881496 10009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a14282e8c84da29c7865f0dc1cce06" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46497 } }
I20260806 16:06:12.883097 10004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:12.883497 10010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69a14282e8c84da29c7865f0dc1cce06. Latest consensus state: current_term: 1 leader_uuid: "69a14282e8c84da29c7865f0dc1cce06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a14282e8c84da29c7865f0dc1cce06" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46497 } } }
I20260806 16:06:12.883658 10010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:12.884099 10014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:12.885555 10011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69a14282e8c84da29c7865f0dc1cce06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a14282e8c84da29c7865f0dc1cce06" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46497 } } }
I20260806 16:06:12.885668 10011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:12.885695 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:12.866626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.8.49.62:46497)
0806 16:06:12.867424 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:12.867448 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:12.867512 (+ 64us) server_negotiation.cc:407] Connection header received
0806 16:06:12.868486 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:12.868511 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:12.869080 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:12.869507 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:12.870567 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:12.871951 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:12.873261 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:12.873891 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:12.874051 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:12.877698 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:12.877705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:12.877725 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:12.877769 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:12.880243 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:12.881021 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:12.881030 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:12.881039 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:12.881209 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:12.881645 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:12.881649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:12.881651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:12.882245 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:12.882439 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:12.883753 (+ 1314us) server_negotiation.cc:299] Negotiation successful
0806 16:06:12.884232 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":368,"threads_started":1}
I20260806 16:06:12.888039 10014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:12.896226 10014 catalog_manager.cc:1347] Generated new cluster ID: ba58ea59999a4f7fa97640df5ddb90f2
I20260806 16:06:12.896323 10014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:12.919203 10014 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:12.920775 10014 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:12.931993 10014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06: Generated new TSK 0
I20260806 16:06:12.933105 10014 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:06:12.985127 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46497
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:13.137013 10036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:13.137451 10036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:13.137508 10036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:13.142531 10036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:13.142679 10036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:13.142733 10036 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:06:13.142773 10036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:13.150342 10036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.49.62:46497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:13.152097 10036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:13.153559 10036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:13.168453 10048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:13.168660 10036 server_base.cc:1034] running on GCE node
W20260806 16:06:13.168450 10049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:13.168469 10051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:13.169507 10036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:13.170456 10036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:13.171743 10036 hybrid_clock.cc:648] HybridClock initialized: now 1786032373171720 us; error 54 us; skew 500 ppm
Aug 06 16:06:13 dist-test-slave-7z32 krb5kdc[9866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:13.175840 10036 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:06:13.177186 10036 webserver.cc:460] Webserver started at http://127.8.49.1:36981/ using document root <none> and password file <none>
I20260806 16:06:13.177981 10036 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:13.178092 10036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:13.178395 10036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:13.181113 10036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "e32b9645dd174e65bb464e5e25079b90"
format_stamp: "Formatted at 2026-08-06 16:06:13 on dist-test-slave-7z32"
server_key: "b72bd751bf19db75bbd57e21e88a9357"
server_key_iv: "5e801d12c2e1c824159caf1d40316850"
server_key_version: "encryptionkey@0"
I20260806 16:06:13.181819 10036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "e32b9645dd174e65bb464e5e25079b90"
format_stamp: "Formatted at 2026-08-06 16:06:13 on dist-test-slave-7z32"
server_key: "b72bd751bf19db75bbd57e21e88a9357"
server_key_iv: "5e801d12c2e1c824159caf1d40316850"
server_key_version: "encryptionkey@0"
I20260806 16:06:13.187230 10036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 16:06:13.190896 10062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:13.192812 10036 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:13.193034 10036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "e32b9645dd174e65bb464e5e25079b90"
format_stamp: "Formatted at 2026-08-06 16:06:13 on dist-test-slave-7z32"
server_key: "b72bd751bf19db75bbd57e21e88a9357"
server_key_iv: "5e801d12c2e1c824159caf1d40316850"
server_key_version: "encryptionkey@0"
I20260806 16:06:13.193192 10036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:13.218245 10036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:13.222631 10036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:13.222926 10036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:13.224812 10036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:13.224874 10036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:13.224924 10036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:13.225008 10036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:13.238816 10036 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:34761
I20260806 16:06:13.239056 10218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:34761 every 8 connection(s)
Aug 06 16:06:13 dist-test-slave-7z32 krb5kdc[9866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:13.240135 10036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:13.245640 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.226428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:43097 (local address 127.8.49.62:46497)
0806 16:06:13.226630 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:13.226637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:13.228865 (+ 2228us) server_negotiation.cc:407] Connection header received
0806 16:06:13.230537 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:13.230545 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:13.230640 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:13.230792 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:13.232877 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.233547 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.236209 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.236631 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.236785 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.240763 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:13.240768 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:13.240771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:13.240814 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:13.242627 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.243376 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.243386 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.243387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.243478 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.243875 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.243878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.243880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.244126 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:13.244241 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:13.245240 (+ 999us) server_negotiation.cc:299] Negotiation successful
0806 16:06:13.245425 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:06:13.246971 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.227301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46497 (local address 127.8.49.1:43097)
0806 16:06:13.228579 (+ 1278us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:13.228668 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:13.230177 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:13.231173 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:13.231185 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:13.231866 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:13.232620 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.232639 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.233730 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.233738 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.235689 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.235704 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.236857 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.236862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.237323 (+ 461us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.238173 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:13.238196 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:13.240445 (+ 2249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:13.242860 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.242869 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.242893 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.243236 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.243582 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.243586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.243588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.243764 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.244242 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:13.244261 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:13.244782 (+ 521us) client_negotiation.cc:769] Sending connection context
0806 16:06:13.245131 (+ 349us) client_negotiation.cc:240] Negotiation successful
0806 16:06:13.245631 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":887,"thread_start_us":300,"threads_started":1}
I20260806 16:06:13.247385 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 10036
I20260806 16:06:13.247531 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:13.247828 8388 external_mini_cluster.cc:1428] Setting key 9d01fd7b9533f15f91ff540bc2a0b97d
I20260806 16:06:13.250047 10219 heartbeater.cc:347] Connected to a master server at 127.8.49.62:46497
I20260806 16:06:13.250403 10219 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:13.251246 10219 heartbeater.cc:511] Master 127.8.49.62:46497 requested a full tablet report, sending...
I20260806 16:06:13.253602 9934 ts_manager.cc:194] Registered new tserver with Master: e32b9645dd174e65bb464e5e25079b90 (127.8.49.1:34761)
I20260806 16:06:13.255383 9934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:43097
I20260806 16:06:13.263588 10228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.253381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46497 (local address 127.8.49.1:53135)
0806 16:06:13.253801 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:13.253819 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:13.254011 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:13.254423 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:13.254426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:13.254729 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:13.254918 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.254927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.256076 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.256079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.256891 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.256901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.257423 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.257426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.257528 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.258226 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:13.258239 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:13.258699 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:13.261311 (+ 2612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.261315 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.261317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.261665 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.262083 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.262086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.262088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.262188 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.262762 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:13.262765 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:13.262978 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 16:06:13.263106 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 16:06:13.263297 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":168,"threads_started":1}
I20260806 16:06:13.263640 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.253450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:53135 (local address 127.8.49.62:46497)
0806 16:06:13.253609 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:13.253614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:13.253918 (+ 304us) server_negotiation.cc:407] Connection header received
0806 16:06:13.254131 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:13.254135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:13.254183 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:13.254240 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:13.255072 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.255834 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.257036 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.257294 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.257408 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.258949 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:13.258955 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:13.258960 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:13.258993 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:13.261123 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.261809 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.261814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.261819 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.261931 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.262319 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.262326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.262330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.262615 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:13.262742 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:13.263209 (+ 467us) server_negotiation.cc:299] Negotiation successful
0806 16:06:13.263415 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:06:13.292515 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46497
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:13.454191 10230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:13.454612 10230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:13.454694 10230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:13.459277 10230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:13.459407 10230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:13.459483 10230 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:06:13.459573 10230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:13.465883 10230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.49.62:46497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:13.467445 10230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:13.468886 10230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:13.483330 10240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:13.483523 10236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:13.483510 10237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:13.483707 10230 server_base.cc:1034] running on GCE node
I20260806 16:06:13.484472 10230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:13.485579 10230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:13.486928 10230 hybrid_clock.cc:648] HybridClock initialized: now 1786032373486918 us; error 58 us; skew 500 ppm
Aug 06 16:06:13 dist-test-slave-7z32 krb5kdc[9866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:13.491056 10230 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:06:13.492347 10230 webserver.cc:460] Webserver started at http://127.8.49.2:33613/ using document root <none> and password file <none>
I20260806 16:06:13.493094 10230 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:13.493171 10230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:13.493402 10230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:13.496313 10230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "435b228f1d394fd6a981d89b8644bd81"
format_stamp: "Formatted at 2026-08-06 16:06:13 on dist-test-slave-7z32"
server_key: "9fec67e21b489553eaa45fda9c9c5530"
server_key_iv: "2510242dbbf83445ea84d81c4dc3dade"
server_key_version: "encryptionkey@0"
I20260806 16:06:13.497128 10230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "435b228f1d394fd6a981d89b8644bd81"
format_stamp: "Formatted at 2026-08-06 16:06:13 on dist-test-slave-7z32"
server_key: "9fec67e21b489553eaa45fda9c9c5530"
server_key_iv: "2510242dbbf83445ea84d81c4dc3dade"
server_key_version: "encryptionkey@0"
I20260806 16:06:13.502678 10230 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:06:13.506295 10247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:13.508368 10230 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:13.508572 10230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "435b228f1d394fd6a981d89b8644bd81"
format_stamp: "Formatted at 2026-08-06 16:06:13 on dist-test-slave-7z32"
server_key: "9fec67e21b489553eaa45fda9c9c5530"
server_key_iv: "2510242dbbf83445ea84d81c4dc3dade"
server_key_version: "encryptionkey@0"
I20260806 16:06:13.508742 10230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:13.531776 10230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:13.536489 10230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:13.536784 10230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:13.539106 10230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:13.539177 10230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:13.539259 10230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:13.539302 10230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:06:13 dist-test-slave-7z32 krb5kdc[9866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:13.555147 10230 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:34453
I20260806 16:06:13.555197 10401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:34453 every 8 connection(s)
I20260806 16:06:13.556521 10230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:13.558878 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.540461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:39491 (local address 127.8.49.62:46497)
0806 16:06:13.540682 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:13.540687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:13.542259 (+ 1572us) server_negotiation.cc:407] Connection header received
0806 16:06:13.543787 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:13.543791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:13.543858 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:13.543979 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:13.545562 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.546258 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.548037 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.548297 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.548423 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.553928 (+ 5505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:13.553933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:13.553937 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:13.553975 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:13.555997 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.556710 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.556714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.556716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.556806 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.557268 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.557272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.557273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.557549 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:13.557674 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:13.558525 (+ 851us) server_negotiation.cc:299] Negotiation successful
0806 16:06:13.558667 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 16:06:13.560032 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.541271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46497 (local address 127.8.49.2:39491)
0806 16:06:13.542047 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:13.542114 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:13.543532 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:13.544203 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:13.544212 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:13.544656 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:13.545371 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.545396 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.546418 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.546424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.547883 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.547895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.548482 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.548486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.548694 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.549547 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:13.549587 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:13.553691 (+ 4104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:13.556147 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.556155 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.556175 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.556517 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.556914 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.556918 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.556920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.557120 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.557726 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:13.557734 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:13.558163 (+ 429us) client_negotiation.cc:769] Sending connection context
0806 16:06:13.558407 (+ 244us) client_negotiation.cc:240] Negotiation successful
0806 16:06:13.558789 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":210,"threads_started":1}
I20260806 16:06:13.561760 10403 heartbeater.cc:347] Connected to a master server at 127.8.49.62:46497
I20260806 16:06:13.561959 10403 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:13.562739 10403 heartbeater.cc:511] Master 127.8.49.62:46497 requested a full tablet report, sending...
I20260806 16:06:13.564429 9930 ts_manager.cc:194] Registered new tserver with Master: 435b228f1d394fd6a981d89b8644bd81 (127.8.49.2:34453)
I20260806 16:06:13.565304 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 10230
I20260806 16:06:13.565433 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:13.565410 9930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:39491
I20260806 16:06:13.565788 8388 external_mini_cluster.cc:1428] Setting key b5c64dc83162bf79c08e75f0b6b67f1a
I20260806 16:06:13.577608 10411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.564852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46497 (local address 127.8.49.2:35913)
0806 16:06:13.566523 (+ 1671us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:13.566539 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:13.567320 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:13.567641 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:13.567644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:13.567959 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:13.568256 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.568269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.569545 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.569552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.570623 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.570637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.571535 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.571540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.571647 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.572349 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:13.572363 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:13.572761 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:13.575449 (+ 2688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.575453 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.575455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.575750 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.576136 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.576139 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.576141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.576236 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.577025 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:13.577028 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:13.577154 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 16:06:13.577262 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 16:06:13.577413 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1538,"thread_start_us":1434,"threads_started":1}
I20260806 16:06:13.577889 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.564947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:35913 (local address 127.8.49.62:46497)
0806 16:06:13.565095 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:13.565100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:13.566630 (+ 1530us) server_negotiation.cc:407] Connection header received
0806 16:06:13.567466 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:13.567469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:13.567527 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:13.567615 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:13.568433 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.569320 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.570869 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.571326 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.571495 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.572946 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:13.572954 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:13.572961 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:13.573000 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:13.575212 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.575880 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.575887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.575892 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.576011 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.576352 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.576358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.576360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.576873 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:13.577015 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:13.577374 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 16:06:13.577565 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:06:13.607457 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46497
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:13.774523 10414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:13.775074 10414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:13.775173 10414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:13.780189 10414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:13.780295 10414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:13.780346 10414 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:06:13.780386 10414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:13.789172 10414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.49.62:46497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:13.791040 10414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:13.792613 10414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:06:13.807134 10414 server_base.cc:1034] running on GCE node
W20260806 16:06:13.806991 10423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:13.806993 10424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:13.807559 10426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:13.808200 10414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:13.809158 10414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:13.810405 10414 hybrid_clock.cc:648] HybridClock initialized: now 1786032373810355 us; error 73 us; skew 500 ppm
Aug 06 16:06:13 dist-test-slave-7z32 krb5kdc[9866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:13.814779 10414 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:06:13.816231 10414 webserver.cc:460] Webserver started at http://127.8.49.3:43889/ using document root <none> and password file <none>
I20260806 16:06:13.817051 10414 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:13.817168 10414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:13.817466 10414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:13.820247 10414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "6de99452d8fc45d396b95fc4bbe0817f"
format_stamp: "Formatted at 2026-08-06 16:06:13 on dist-test-slave-7z32"
server_key: "07b2e7940cdf23958e2455ea0b9854c4"
server_key_iv: "58c551675a5a30884a7131711e6dbce2"
server_key_version: "encryptionkey@0"
I20260806 16:06:13.820932 10414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "6de99452d8fc45d396b95fc4bbe0817f"
format_stamp: "Formatted at 2026-08-06 16:06:13 on dist-test-slave-7z32"
server_key: "07b2e7940cdf23958e2455ea0b9854c4"
server_key_iv: "58c551675a5a30884a7131711e6dbce2"
server_key_version: "encryptionkey@0"
I20260806 16:06:13.826090 10414 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:06:13.829840 10436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:13.831836 10414 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:13.832063 10414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "6de99452d8fc45d396b95fc4bbe0817f"
format_stamp: "Formatted at 2026-08-06 16:06:13 on dist-test-slave-7z32"
server_key: "07b2e7940cdf23958e2455ea0b9854c4"
server_key_iv: "58c551675a5a30884a7131711e6dbce2"
server_key_version: "encryptionkey@0"
I20260806 16:06:13.832211 10414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:13.854607 10414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:13.858745 10414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:13.859016 10414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:13.861074 10414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:13.861151 10414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:13.861208 10414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:13.861243 10414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:13.873487 10414 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:46429
I20260806 16:06:13.873890 10590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:46429 every 8 connection(s)
Aug 06 16:06:13 dist-test-slave-7z32 krb5kdc[9866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:13.874797 10414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:13.880190 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.862684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:55827 (local address 127.8.49.62:46497)
0806 16:06:13.862901 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:13.862907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:13.864370 (+ 1463us) server_negotiation.cc:407] Connection header received
0806 16:06:13.865950 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:13.865956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:13.866044 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:13.866119 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:13.868638 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.869341 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.870817 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.871080 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.871223 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.875023 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:13.875027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:13.875030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:13.875067 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:13.876974 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.877804 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.877807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.877809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.877940 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.878508 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.878515 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.878520 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.878881 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:13.878994 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:13.879733 (+ 739us) server_negotiation.cc:299] Negotiation successful
0806 16:06:13.879907 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 16:06:13.880450 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 10414
I20260806 16:06:13.880589 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:13.880959 8388 external_mini_cluster.cc:1428] Setting key 2d98cdbe26f509bfa40e7fc021b27eee
I20260806 16:06:13.881227 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.863313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46497 (local address 127.8.49.3:55827)
0806 16:06:13.864146 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:13.864214 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:13.865577 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:13.866725 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:13.866738 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:13.867448 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:13.868419 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.868446 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.869503 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.869507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.870671 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.870683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.871291 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.871295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.871507 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.872426 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:13.872453 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:13.874677 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:13.877170 (+ 2493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.877178 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.877193 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.877630 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.878099 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.878102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.878104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.878326 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.879030 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:13.879037 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:13.879387 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 16:06:13.879597 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 16:06:13.879950 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":215,"threads_started":1}
I20260806 16:06:13.883387 10591 heartbeater.cc:347] Connected to a master server at 127.8.49.62:46497
I20260806 16:06:13.883625 10591 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:13.884552 10591 heartbeater.cc:511] Master 127.8.49.62:46497 requested a full tablet report, sending...
I20260806 16:06:13.886132 9930 ts_manager.cc:194] Registered new tserver with Master: 6de99452d8fc45d396b95fc4bbe0817f (127.8.49.3:46429)
I20260806 16:06:13.887070 9930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:55827
I20260806 16:06:13.896539 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.886953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46497 (local address 127.8.49.3:35087)
0806 16:06:13.887311 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:13.887327 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:13.887493 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:13.887848 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:13.887851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:13.888222 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:13.888370 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.888379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.889346 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.889350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.890232 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.890243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.890748 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.890751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.890909 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.891567 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:13.891581 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:13.892054 (+ 473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:13.894249 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.894252 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.894253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.894604 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.895082 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:13.895085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:13.895086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:13.895243 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:13.895863 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:13.895866 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:13.896000 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 16:06:13.896120 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 16:06:13.896335 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":167,"threads_started":1}
I20260806 16:06:13.896680 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.886788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:35087 (local address 127.8.49.62:46497)
0806 16:06:13.887000 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:13.887007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:13.887424 (+ 417us) server_negotiation.cc:407] Connection header received
0806 16:06:13.887626 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:13.887630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:13.887690 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:13.887828 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:13.888546 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.889131 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.890372 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.890618 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.890717 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.892231 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:13.892234 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:13.892237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:13.892264 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:13.894094 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.894771 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.894776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.894779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.894934 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.895396 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.895399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.895401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.895724 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:13.895856 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:13.896232 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 16:06:13.896451 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 16:06:13.897603 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 16:06:13 dist-test-slave-7z32 krb5kdc[9866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 16:06:13 dist-test-slave-7z32 krb5kdc[9866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:13.936702 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.924305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.8.49.62:46497)
0806 16:06:13.924469 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:13.924474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:13.924539 (+ 65us) server_negotiation.cc:407] Connection header received
0806 16:06:13.924774 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:13.924778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:13.924853 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:13.924972 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:13.925919 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.926561 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.927576 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.927813 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.927948 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.931602 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:13.931607 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:13.931609 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:13.931659 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:13.934109 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.934774 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.934780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.934782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.934989 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:13.935382 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:13.935386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:13.935389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:13.935752 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:13.935938 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:13.936288 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 16:06:13.936454 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:06:13.941386 9934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:13.945626 9934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:13.965622 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.958195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:34761 (local address 127.0.0.1:58698)
0806 16:06:13.959770 (+ 1575us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:13.959786 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:13.959884 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:13.960540 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:13.960546 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:13.960570 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:13.960679 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.960685 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.962218 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.962224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.963876 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.963892 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.965229 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.965232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.965314 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.965337 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:06:13.965406 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 16:06:13.965462 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1505,"mutex_wait_us":23,"thread_start_us":138,"threads_started":1}
I20260806 16:06:13.965622 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.958061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:46429 (local address 127.0.0.1:44854)
0806 16:06:13.958778 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:13.958898 (+ 120us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:13.959096 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:13.960130 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:13.960141 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:13.960165 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:13.960321 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.960326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.961853 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.962040 (+ 187us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.963605 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.963617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.964873 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.964877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.965027 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.965074 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 16:06:13.965216 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 16:06:13.965396 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":55,"thread_start_us":137,"threads_started":1}
I20260806 16:06:13.966127 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.958397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44854 (local address 127.8.49.3:46429)
0806 16:06:13.959543 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:13.959558 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:13.959591 (+ 33us) server_negotiation.cc:407] Connection header received
0806 16:06:13.959699 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:13.959705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:13.959944 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:13.960125 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:13.960482 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.961699 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.963772 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.964682 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.964820 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.965058 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:13.965781 (+ 723us) server_negotiation.cc:299] Negotiation successful
0806 16:06:13.965921 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":183,"threads_started":1}
I20260806 16:06:13.966632 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.959155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58698 (local address 127.8.49.1:34761)
0806 16:06:13.960140 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:13.960147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:13.960167 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:06:13.960242 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:13.960247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:13.960416 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:13.960565 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:13.960812 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.962066 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.964095 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.964861 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.964982 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.965161 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:13.966335 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0806 16:06:13.966460 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":882,"thread_start_us":99,"threads_started":1}
I20260806 16:06:13.966609 10617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.959572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40032 (local address 127.8.49.2:34453)
0806 16:06:13.960547 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:13.960560 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:13.960584 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:06:13.960658 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:13.960663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:13.960862 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:13.961035 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:13.961302 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.962507 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.964070 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.964953 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.965086 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.965280 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:13.966355 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0806 16:06:13.966479 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":179,"threads_started":1}
I20260806 16:06:13.967064 10612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:13.959339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:34453 (local address 127.0.0.1:40032)
0806 16:06:13.959819 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:13.959836 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:13.959941 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:13.961025 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:13.961029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:13.961049 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:13.961167 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.961173 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.962646 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.962650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.963918 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:13.963927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:13.965364 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:13.965368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:13.965482 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:13.965525 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 16:06:13.965605 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 16:06:13.966918 (+ 1313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":5,"thread_start_us":108,"threads_started":1}
I20260806 16:06:13.970203 10508 tablet_service.cc:1467] Processing CreateTablet for tablet cac2d4ff62054611abbb87af380bf8ab (DEFAULT_TABLE table=test-table [id=98c7d360792b471eaba9df68d8146f36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:13.970544 10132 tablet_service.cc:1467] Processing CreateTablet for tablet cac2d4ff62054611abbb87af380bf8ab (DEFAULT_TABLE table=test-table [id=98c7d360792b471eaba9df68d8146f36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:13.970547 10309 tablet_service.cc:1467] Processing CreateTablet for tablet cac2d4ff62054611abbb87af380bf8ab (DEFAULT_TABLE table=test-table [id=98c7d360792b471eaba9df68d8146f36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:13.971575 10508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac2d4ff62054611abbb87af380bf8ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:13.971951 10309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac2d4ff62054611abbb87af380bf8ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:13.972021 10132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac2d4ff62054611abbb87af380bf8ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:13.980232 10618 tablet_bootstrap.cc:492] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f: Bootstrap starting.
I20260806 16:06:13.982195 10619 tablet_bootstrap.cc:492] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90: Bootstrap starting.
I20260806 16:06:13.983459 10620 tablet_bootstrap.cc:492] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: Bootstrap starting.
I20260806 16:06:13.984545 10618 tablet_bootstrap.cc:654] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:13.985574 10618 log.cc:824] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:13.986629 10619 tablet_bootstrap.cc:654] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:13.987522 10620 tablet_bootstrap.cc:654] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:13.987622 10619 log.cc:824] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:13.988394 10620 log.cc:824] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:13.989840 10618 tablet_bootstrap.cc:492] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f: No bootstrap required, opened a new log
I20260806 16:06:13.990101 10618 ts_tablet_manager.cc:1397] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260806 16:06:13.990749 10619 tablet_bootstrap.cc:492] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90: No bootstrap required, opened a new log
I20260806 16:06:13.990952 10620 tablet_bootstrap.cc:492] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: No bootstrap required, opened a new log
I20260806 16:06:13.991168 10620 ts_tablet_manager.cc:1397] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 16:06:13.991539 10619 ts_tablet_manager.cc:1397] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.002s
I20260806 16:06:13.994694 10618 raft_consensus.cc:348] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } }
I20260806 16:06:13.995070 10618 raft_consensus.cc:374] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:13.995148 10618 raft_consensus.cc:729] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de99452d8fc45d396b95fc4bbe0817f, State: Initialized, Role: FOLLOWER
I20260806 16:06:13.995236 10620 raft_consensus.cc:348] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } }
I20260806 16:06:13.995481 10620 raft_consensus.cc:374] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:13.995528 10620 raft_consensus.cc:729] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435b228f1d394fd6a981d89b8644bd81, State: Initialized, Role: FOLLOWER
I20260806 16:06:13.995695 10618 consensus_queue.cc:260] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } }
I20260806 16:06:13.995976 10620 consensus_queue.cc:260] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } }
I20260806 16:06:13.997148 10618 ts_tablet_manager.cc:1428] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:06:13.997120 10403 heartbeater.cc:503] Master 127.8.49.62:46497 was elected leader, sending a full tablet report...
I20260806 16:06:13.997176 10620 ts_tablet_manager.cc:1428] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 16:06:13.997200 10619 raft_consensus.cc:348] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } }
I20260806 16:06:13.997648 10619 raft_consensus.cc:374] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:13.997646 10591 heartbeater.cc:503] Master 127.8.49.62:46497 was elected leader, sending a full tablet report...
I20260806 16:06:13.997728 10619 raft_consensus.cc:729] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32b9645dd174e65bb464e5e25079b90, State: Initialized, Role: FOLLOWER
I20260806 16:06:13.998353 10619 consensus_queue.cc:260] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } }
I20260806 16:06:13.999343 10219 heartbeater.cc:503] Master 127.8.49.62:46497 was elected leader, sending a full tablet report...
I20260806 16:06:13.999887 10619 ts_tablet_manager.cc:1428] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260806 16:06:14.046810 10629 raft_consensus.cc:482] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:14.047186 10629 raft_consensus.cc:504] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } }
I20260806 16:06:14.048676 10629 leader_election.cc:288] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6de99452d8fc45d396b95fc4bbe0817f (127.8.49.3:46429), 435b228f1d394fd6a981d89b8644bd81 (127.8.49.2:34453)
I20260806 16:06:14.054306 10617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:14.050233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:46521 (local address 127.8.49.2:34453)
0806 16:06:14.050502 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:14.050508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:14.050675 (+ 167us) server_negotiation.cc:407] Connection header received
0806 16:06:14.050939 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:14.050942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:14.050990 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:14.051091 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:14.051498 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.052050 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.053243 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.053666 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.053787 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:14.053861 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:14.054068 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 16:06:14.054144 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 16:06:14.054388 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:14.049384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:35273 (local address 127.8.49.3:46429)
0806 16:06:14.049608 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:14.049612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:14.049926 (+ 314us) server_negotiation.cc:407] Connection header received
0806 16:06:14.050295 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:14.050301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:14.050725 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:14.050814 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:14.051498 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.052148 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.053289 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.053712 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.053827 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:14.053916 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:14.054115 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 16:06:14.054216 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260806 16:06:14.054418 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:14.049229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:46429 (local address 127.8.49.1:35273)
0806 16:06:14.049708 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:14.049728 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:14.050080 (+ 352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:14.051104 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:14.051107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:14.051132 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:14.051356 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:14.051363 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.052275 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.052282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:14.053141 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:14.053151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.053845 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.053849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:14.053978 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:14.054000 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:06:14.054095 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 16:06:14.054194 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":34,"thread_start_us":210,"threads_started":1}
I20260806 16:06:14.054813 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:14.049912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:34453 (local address 127.8.49.1:46521)
0806 16:06:14.050525 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:14.050556 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:14.050765 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:14.051092 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:14.051095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:14.051113 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:14.051374 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:14.051381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.052170 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.052177 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:14.053105 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:14.053115 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.053798 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.053801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:14.053913 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:14.053952 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 16:06:14.054048 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 16:06:14.054144 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":32,"thread_start_us":158,"threads_started":1}
I20260806 16:06:14.055436 10535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac2d4ff62054611abbb87af380bf8ab" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de99452d8fc45d396b95fc4bbe0817f" is_pre_election: true
I20260806 16:06:14.055581 10338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac2d4ff62054611abbb87af380bf8ab" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435b228f1d394fd6a981d89b8644bd81" is_pre_election: true
I20260806 16:06:14.055912 10338 raft_consensus.cc:2454] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 0.
I20260806 16:06:14.055912 10535 raft_consensus.cc:2454] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 0.
I20260806 16:06:14.056529 10064 leader_election.cc:302] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 435b228f1d394fd6a981d89b8644bd81, e32b9645dd174e65bb464e5e25079b90; no voters:
I20260806 16:06:14.056890 10629 raft_consensus.cc:2781] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:14.057003 10629 raft_consensus.cc:482] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:14.057037 10629 raft_consensus.cc:3037] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Advancing to term 1
W20260806 16:06:14.058121 10404 tablet.cc:2369] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:14.058593 10629 raft_consensus.cc:504] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } }
I20260806 16:06:14.059089 10629 leader_election.cc:288] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 election: Requested vote from peers 6de99452d8fc45d396b95fc4bbe0817f (127.8.49.3:46429), 435b228f1d394fd6a981d89b8644bd81 (127.8.49.2:34453)
I20260806 16:06:14.059682 10535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac2d4ff62054611abbb87af380bf8ab" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de99452d8fc45d396b95fc4bbe0817f"
I20260806 16:06:14.059726 10338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac2d4ff62054611abbb87af380bf8ab" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435b228f1d394fd6a981d89b8644bd81"
I20260806 16:06:14.059818 10535 raft_consensus.cc:3037] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:14.059981 10338 raft_consensus.cc:3037] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:14.061534 10338 raft_consensus.cc:2454] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 1.
I20260806 16:06:14.061558 10535 raft_consensus.cc:2454] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 1.
I20260806 16:06:14.061992 10064 leader_election.cc:302] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 435b228f1d394fd6a981d89b8644bd81, e32b9645dd174e65bb464e5e25079b90; no voters:
I20260806 16:06:14.062280 10629 raft_consensus.cc:2781] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:14.062641 10629 raft_consensus.cc:686] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 1 LEADER]: Becoming Leader. State: Replica: e32b9645dd174e65bb464e5e25079b90, State: Running, Role: LEADER
I20260806 16:06:14.063210 10629 consensus_queue.cc:237] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } }
I20260806 16:06:14.067729 9934 catalog_manager.cc:5495] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 reported cstate change: term changed from 0 to 1, leader changed from <none> to e32b9645dd174e65bb464e5e25079b90 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "e32b9645dd174e65bb464e5e25079b90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } health_report { overall_health: UNKNOWN } } }
I20260806 16:06:14.104707 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:14.100180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.8.49.1:34761)
0806 16:06:14.100381 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:14.100385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:14.100409 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:06:14.100673 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:14.100675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:14.100740 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:14.100874 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:14.101290 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.101833 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.103000 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.103233 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.103329 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:14.103706 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:14.104216 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:14.104321 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:14.104522 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 16:06:14.104583 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:06:14.113003 10338 raft_consensus.cc:1261] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81 [term 1 FOLLOWER]: Refusing update from remote peer e32b9645dd174e65bb464e5e25079b90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:14.113561 10535 raft_consensus.cc:1261] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f [term 1 FOLLOWER]: Refusing update from remote peer e32b9645dd174e65bb464e5e25079b90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:14.114208 10635 consensus_queue.cc:1035] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:14.114514 10629 consensus_queue.cc:1035] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:14.126011 10642 mvcc.cc:205] Tried to move back new op lower bound from 7315588604356149248 to 7315588604166631424. Current Snapshot: MvccSnapshot[applied={T|T < 7315588604356149248}]
I20260806 16:06:14.129220 10641 mvcc.cc:205] Tried to move back new op lower bound from 7315588604356149248 to 7315588604166631424. Current Snapshot: MvccSnapshot[applied={T|T < 7315588604356149248 or (T in {7315588604356149248})}]
I20260806 16:06:14.141510 9934 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:14.141889 9934 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:14.145267 9934 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06: Sending DeleteTablet for 3 replicas of tablet cac2d4ff62054611abbb87af380bf8ab
I20260806 16:06:14.146502 10309 tablet_service.cc:1514] Processing DeleteTablet for tablet cac2d4ff62054611abbb87af380bf8ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:14 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:40032
I20260806 16:06:14.146696 10508 tablet_service.cc:1514] Processing DeleteTablet for tablet cac2d4ff62054611abbb87af380bf8ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:14 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:44854
I20260806 16:06:14.146621 10132 tablet_service.cc:1514] Processing DeleteTablet for tablet cac2d4ff62054611abbb87af380bf8ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:14 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:58698
I20260806 16:06:14.147220 10654 tablet_replica.cc:330] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f: stopping tablet replica
I20260806 16:06:14.147636 10654 raft_consensus.cc:2229] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:14.147702 10657 tablet_replica.cc:330] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90: stopping tablet replica
I20260806 16:06:14.148059 10654 raft_consensus.cc:2258] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:14.148341 10657 raft_consensus.cc:2229] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:14.148360 9934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:14.148887 9934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:14.149461 10657 raft_consensus.cc:2258] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:14.149849 10653 tablet_replica.cc:330] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: stopping tablet replica
I20260806 16:06:14.150223 10653 raft_consensus.cc:2229] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:14.150632 10653 raft_consensus.cc:2258] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:14.151547 10654 ts_tablet_manager.cc:1905] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:14.151952 10657 ts_tablet_manager.cc:1905] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:14.154942 10654 ts_tablet_manager.cc:1918] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:14.155092 10654 log.cc:1196] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/cac2d4ff62054611abbb87af380bf8ab
I20260806 16:06:14.155503 10654 ts_tablet_manager.cc:1939] T cac2d4ff62054611abbb87af380bf8ab P 6de99452d8fc45d396b95fc4bbe0817f: Deleting consensus metadata
I20260806 16:06:14.156172 10657 ts_tablet_manager.cc:1918] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:14.156391 10657 log.cc:1196] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/wal/wals/cac2d4ff62054611abbb87af380bf8ab
I20260806 16:06:14.157071 10309 tablet_service.cc:1467] Processing CreateTablet for tablet ef26e69f527943a48fe579da65256dca (DEFAULT_TABLE table=test-table [id=486391bc1a724733882c45ee5943d4fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:14.157122 10132 tablet_service.cc:1467] Processing CreateTablet for tablet ef26e69f527943a48fe579da65256dca (DEFAULT_TABLE table=test-table [id=486391bc1a724733882c45ee5943d4fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:14.157172 10508 tablet_service.cc:1467] Processing CreateTablet for tablet ef26e69f527943a48fe579da65256dca (DEFAULT_TABLE table=test-table [id=486391bc1a724733882c45ee5943d4fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:14.157418 10508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef26e69f527943a48fe579da65256dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:14.157430 10309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef26e69f527943a48fe579da65256dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:14.157460 10132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef26e69f527943a48fe579da65256dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:14.160077 10618 tablet_bootstrap.cc:492] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f: Bootstrap starting.
I20260806 16:06:14.160480 10619 tablet_bootstrap.cc:492] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90: Bootstrap starting.
I20260806 16:06:14.161044 10620 tablet_bootstrap.cc:492] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81: Bootstrap starting.
I20260806 16:06:14.161229 10618 tablet_bootstrap.cc:654] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:14.161854 10619 tablet_bootstrap.cc:654] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:14.162171 10620 tablet_bootstrap.cc:654] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:14.163218 9909 catalog_manager.cc:4841] TS 6de99452d8fc45d396b95fc4bbe0817f (127.8.49.3:46429): tablet cac2d4ff62054611abbb87af380bf8ab (table test-table [id=98c7d360792b471eaba9df68d8146f36]) successfully deleted
I20260806 16:06:14.163909 10657 ts_tablet_manager.cc:1939] T cac2d4ff62054611abbb87af380bf8ab P e32b9645dd174e65bb464e5e25079b90: Deleting consensus metadata
I20260806 16:06:14.164747 10653 ts_tablet_manager.cc:1905] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:14.165637 10620 tablet_bootstrap.cc:492] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81: No bootstrap required, opened a new log
I20260806 16:06:14.165766 10620 ts_tablet_manager.cc:1397] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 16:06:14.165923 9910 catalog_manager.cc:4841] TS e32b9645dd174e65bb464e5e25079b90 (127.8.49.1:34761): tablet cac2d4ff62054611abbb87af380bf8ab (table test-table [id=98c7d360792b471eaba9df68d8146f36]) successfully deleted
I20260806 16:06:14.166267 10620 raft_consensus.cc:348] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.166415 10620 raft_consensus.cc:374] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:14.166460 10620 raft_consensus.cc:729] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435b228f1d394fd6a981d89b8644bd81, State: Initialized, Role: FOLLOWER
I20260806 16:06:14.166627 10620 consensus_queue.cc:260] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.166965 10620 ts_tablet_manager.cc:1428] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:06:14.167909 10653 ts_tablet_manager.cc:1918] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:14.168088 10619 tablet_bootstrap.cc:492] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90: No bootstrap required, opened a new log
I20260806 16:06:14.168278 10619 ts_tablet_manager.cc:1397] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260806 16:06:14.168956 10619 raft_consensus.cc:348] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.169106 10619 raft_consensus.cc:374] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:14.169152 10619 raft_consensus.cc:729] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32b9645dd174e65bb464e5e25079b90, State: Initialized, Role: FOLLOWER
I20260806 16:06:14.168025 10653 log.cc:1196] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/cac2d4ff62054611abbb87af380bf8ab
I20260806 16:06:14.169440 10619 consensus_queue.cc:260] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.169719 10653 ts_tablet_manager.cc:1939] T cac2d4ff62054611abbb87af380bf8ab P 435b228f1d394fd6a981d89b8644bd81: Deleting consensus metadata
I20260806 16:06:14.169880 10619 ts_tablet_manager.cc:1428] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:14.169965 10618 tablet_bootstrap.cc:492] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f: No bootstrap required, opened a new log
I20260806 16:06:14.170064 10618 ts_tablet_manager.cc:1397] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260806 16:06:14.170534 10618 raft_consensus.cc:348] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.170675 10618 raft_consensus.cc:374] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:14.170711 10618 raft_consensus.cc:729] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de99452d8fc45d396b95fc4bbe0817f, State: Initialized, Role: FOLLOWER
I20260806 16:06:14.170889 10618 consensus_queue.cc:260] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.171193 10618 ts_tablet_manager.cc:1428] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:06:14.171267 9909 catalog_manager.cc:4841] TS 435b228f1d394fd6a981d89b8644bd81 (127.8.49.2:34453): tablet cac2d4ff62054611abbb87af380bf8ab (table test-table [id=98c7d360792b471eaba9df68d8146f36]) successfully deleted
W20260806 16:06:14.242376 10220 tablet.cc:2369] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:06:14.377614 10592 tablet.cc:2369] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:14.452255 10647 raft_consensus.cc:482] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:14.452565 10647 raft_consensus.cc:504] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.453095 10647 leader_election.cc:288] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 435b228f1d394fd6a981d89b8644bd81 (127.8.49.2:34453), 6de99452d8fc45d396b95fc4bbe0817f (127.8.49.3:46429)
I20260806 16:06:14.453868 10535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef26e69f527943a48fe579da65256dca" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de99452d8fc45d396b95fc4bbe0817f" is_pre_election: true
I20260806 16:06:14.454074 10535 raft_consensus.cc:2454] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 0.
I20260806 16:06:14.454015 10338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef26e69f527943a48fe579da65256dca" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435b228f1d394fd6a981d89b8644bd81" is_pre_election: true
I20260806 16:06:14.454528 10338 raft_consensus.cc:2454] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 0.
I20260806 16:06:14.454586 10064 leader_election.cc:302] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de99452d8fc45d396b95fc4bbe0817f, e32b9645dd174e65bb464e5e25079b90; no voters:
I20260806 16:06:14.454869 10647 raft_consensus.cc:2781] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:14.454965 10647 raft_consensus.cc:482] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:14.455010 10647 raft_consensus.cc:3037] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:14.456427 10647 raft_consensus.cc:504] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.456911 10647 leader_election.cc:288] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 election: Requested vote from peers 435b228f1d394fd6a981d89b8644bd81 (127.8.49.2:34453), 6de99452d8fc45d396b95fc4bbe0817f (127.8.49.3:46429)
I20260806 16:06:14.457571 10535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef26e69f527943a48fe579da65256dca" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de99452d8fc45d396b95fc4bbe0817f"
I20260806 16:06:14.457732 10535 raft_consensus.cc:3037] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:14.457633 10338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef26e69f527943a48fe579da65256dca" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435b228f1d394fd6a981d89b8644bd81"
I20260806 16:06:14.457859 10338 raft_consensus.cc:3037] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:14.459101 10535 raft_consensus.cc:2454] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 1.
I20260806 16:06:14.459494 10338 raft_consensus.cc:2454] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 1.
I20260806 16:06:14.459494 10064 leader_election.cc:302] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de99452d8fc45d396b95fc4bbe0817f, e32b9645dd174e65bb464e5e25079b90; no voters:
I20260806 16:06:14.459753 10647 raft_consensus.cc:2781] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:14.459950 10647 raft_consensus.cc:686] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 1 LEADER]: Becoming Leader. State: Replica: e32b9645dd174e65bb464e5e25079b90, State: Running, Role: LEADER
I20260806 16:06:14.460150 10647 consensus_queue.cc:237] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.462553 9930 catalog_manager.cc:5495] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 reported cstate change: term changed from 0 to 1, leader changed from <none> to e32b9645dd174e65bb464e5e25079b90 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "e32b9645dd174e65bb464e5e25079b90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } health_report { overall_health: UNKNOWN } } }
Aug 06 16:06:14 dist-test-slave-7z32 krb5kdc[9866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032372, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:14.579051 10670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:14.565686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46497 (local address 127.0.0.1:55990)
0806 16:06:14.566185 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:14.566211 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:14.566373 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:14.566774 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:14.566777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:14.567468 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:14.567615 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:14.567623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.568865 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.568868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:14.569572 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:14.569584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.570487 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.570490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:14.570581 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:14.571890 (+ 1309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:14.571908 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:14.573848 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:14.576574 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:14.576590 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:14.576600 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:14.577024 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:14.577460 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:14.577462 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:14.577464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:14.577566 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:14.578130 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:14.578139 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:14.578351 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 16:06:14.578483 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 16:06:14.578708 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":169,"threads_started":1}
I20260806 16:06:14.579141 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:14.565690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55990 (local address 127.8.49.62:46497)
0806 16:06:14.566161 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:14.566171 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:14.566312 (+ 141us) server_negotiation.cc:407] Connection header received
0806 16:06:14.566489 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:14.566493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:14.566559 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:14.566708 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:14.567773 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.568686 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.569743 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.570282 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.570436 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:14.574093 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:14.574097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:14.574099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:14.574137 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:14.576363 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:14.577205 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:14.577209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:14.577211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:14.577335 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:14.577701 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:14.577705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:14.577706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:14.577957 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:14.578067 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:14.578601 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 16:06:14.578797 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":353,"thread_start_us":182,"threads_started":1}
I20260806 16:06:14.585682 9930 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:55990:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 16:06:14.587291 9930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:14.594090 10132 tablet_service.cc:1467] Processing CreateTablet for tablet ce1707af045046d98fc8ddde2f7bb803 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c8af2d68f6834dc58a577779d8ece02d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:06:14.594090 10508 tablet_service.cc:1467] Processing CreateTablet for tablet ce1707af045046d98fc8ddde2f7bb803 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c8af2d68f6834dc58a577779d8ece02d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:06:14.594167 10309 tablet_service.cc:1467] Processing CreateTablet for tablet ce1707af045046d98fc8ddde2f7bb803 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c8af2d68f6834dc58a577779d8ece02d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:06:14.594626 10132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce1707af045046d98fc8ddde2f7bb803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:14.594626 10508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce1707af045046d98fc8ddde2f7bb803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:14.594931 10309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce1707af045046d98fc8ddde2f7bb803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:14.598129 10619 tablet_bootstrap.cc:492] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: Bootstrap starting.
I20260806 16:06:14.598405 10618 tablet_bootstrap.cc:492] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f: Bootstrap starting.
I20260806 16:06:14.599392 10619 tablet_bootstrap.cc:654] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:14.599715 10620 tablet_bootstrap.cc:492] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81: Bootstrap starting.
I20260806 16:06:14.600021 10618 tablet_bootstrap.cc:654] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:14.600848 10620 tablet_bootstrap.cc:654] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:14.601022 10619 tablet_bootstrap.cc:492] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: No bootstrap required, opened a new log
I20260806 16:06:14.601171 10619 ts_tablet_manager.cc:1397] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:06:14.601859 10619 raft_consensus.cc:348] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.602167 10619 raft_consensus.cc:374] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:14.602290 10619 raft_consensus.cc:729] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32b9645dd174e65bb464e5e25079b90, State: Initialized, Role: FOLLOWER
I20260806 16:06:14.602528 10618 tablet_bootstrap.cc:492] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f: No bootstrap required, opened a new log
I20260806 16:06:14.602481 10619 consensus_queue.cc:260] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.602861 10618 ts_tablet_manager.cc:1397] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 16:06:14.602938 10620 tablet_bootstrap.cc:492] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81: No bootstrap required, opened a new log
I20260806 16:06:14.603073 10620 ts_tablet_manager.cc:1397] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260806 16:06:14.603268 10619 ts_tablet_manager.cc:1428] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 16:06:14.603581 10618 raft_consensus.cc:348] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.603574 10647 tablet_replica.cc:439] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } }
I20260806 16:06:14.603732 10618 raft_consensus.cc:374] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:14.603770 10618 raft_consensus.cc:729] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de99452d8fc45d396b95fc4bbe0817f, State: Initialized, Role: FOLLOWER
I20260806 16:06:14.603832 10647 tablet_replica.cc:442] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:14.603999 10618 consensus_queue.cc:260] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.604499 10627 tablet_replica.cc:439] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } }
I20260806 16:06:14.604696 10627 tablet_replica.cc:442] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:14.605255 10618 ts_tablet_manager.cc:1428] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:14.604809 10620 raft_consensus.cc:348] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.605726 10620 raft_consensus.cc:374] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:14.605804 10620 raft_consensus.cc:729] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435b228f1d394fd6a981d89b8644bd81, State: Initialized, Role: FOLLOWER
I20260806 16:06:14.606006 10620 consensus_queue.cc:260] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.606439 10620 ts_tablet_manager.cc:1428] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 16:06:14.606387 10628 tablet_replica.cc:439] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } }
I20260806 16:06:14.606559 10628 tablet_replica.cc:442] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:14.831612 10647 raft_consensus.cc:482] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:14.831872 10647 raft_consensus.cc:504] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.832545 10647 leader_election.cc:288] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 435b228f1d394fd6a981d89b8644bd81 (127.8.49.2:34453), 6de99452d8fc45d396b95fc4bbe0817f (127.8.49.3:46429)
I20260806 16:06:14.833542 10338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce1707af045046d98fc8ddde2f7bb803" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435b228f1d394fd6a981d89b8644bd81" is_pre_election: true
I20260806 16:06:14.833549 10535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce1707af045046d98fc8ddde2f7bb803" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de99452d8fc45d396b95fc4bbe0817f" is_pre_election: true
I20260806 16:06:14.833796 10535 raft_consensus.cc:2454] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 0.
I20260806 16:06:14.833796 10338 raft_consensus.cc:2454] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 0.
I20260806 16:06:14.834460 10064 leader_election.cc:302] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de99452d8fc45d396b95fc4bbe0817f, e32b9645dd174e65bb464e5e25079b90; no voters:
I20260806 16:06:14.834777 10647 raft_consensus.cc:2781] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:14.834888 10647 raft_consensus.cc:482] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:14.834952 10647 raft_consensus.cc:3037] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:14.836428 10647 raft_consensus.cc:504] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.836982 10647 leader_election.cc:288] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 election: Requested vote from peers 435b228f1d394fd6a981d89b8644bd81 (127.8.49.2:34453), 6de99452d8fc45d396b95fc4bbe0817f (127.8.49.3:46429)
I20260806 16:06:14.837783 10338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce1707af045046d98fc8ddde2f7bb803" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435b228f1d394fd6a981d89b8644bd81"
I20260806 16:06:14.837936 10338 raft_consensus.cc:3037] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:14.837927 10535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce1707af045046d98fc8ddde2f7bb803" candidate_uuid: "e32b9645dd174e65bb464e5e25079b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6de99452d8fc45d396b95fc4bbe0817f"
I20260806 16:06:14.838200 10535 raft_consensus.cc:3037] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:14.839672 10338 raft_consensus.cc:2454] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 1.
I20260806 16:06:14.839885 10535 raft_consensus.cc:2454] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e32b9645dd174e65bb464e5e25079b90 in term 1.
I20260806 16:06:14.840415 10064 leader_election.cc:302] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 435b228f1d394fd6a981d89b8644bd81, e32b9645dd174e65bb464e5e25079b90; no voters:
I20260806 16:06:14.840760 10647 raft_consensus.cc:2781] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:14.840965 10647 raft_consensus.cc:686] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [term 1 LEADER]: Becoming Leader. State: Replica: e32b9645dd174e65bb464e5e25079b90, State: Running, Role: LEADER
I20260806 16:06:14.841169 10647 consensus_queue.cc:237] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } }
I20260806 16:06:14.842363 10635 tablet_replica.cc:439] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: TxnStatusTablet state changed. Reason: New leader e32b9645dd174e65bb464e5e25079b90. Latest consensus state: current_term: 1 leader_uuid: "e32b9645dd174e65bb464e5e25079b90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } }
I20260806 16:06:14.842517 10635 tablet_replica.cc:442] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:06:14.842799 10677 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 16:06:14.843041 10677 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 16:06:14.843680 9930 catalog_manager.cc:5495] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 reported cstate change: term changed from 0 to 1, leader changed from <none> to e32b9645dd174e65bb464e5e25079b90 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "e32b9645dd174e65bb464e5e25079b90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } health_report { overall_health: UNKNOWN } } }
I20260806 16:06:14.885679 10635 consensus_queue.cc:1035] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:14.892814 10647 consensus_queue.cc:1035] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:14.908958 10670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:14.904381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:34761 (local address 127.0.0.1:58726)
0806 16:06:14.904561 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:14.904581 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:14.904712 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:14.905382 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:14.905385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:14.905396 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:06:14.905523 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:14.905528 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.906627 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.906634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:14.907545 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:14.907557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.908089 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.908092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:14.908181 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:14.908245 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:06:14.908589 (+ 344us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:06:14.908611 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:06:14.908698 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 16:06:14.908802 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 16:06:14.909018 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:14.904597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58726 (local address 127.8.49.1:34761)
0806 16:06:14.905115 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:14.905123 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:14.905138 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:06:14.905191 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:14.905194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:14.905236 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:14.905342 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:14.905649 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.906482 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.907701 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.907961 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.908078 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:14.908405 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:14.908499 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:14.908582 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:14.908714 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 16:06:14.908777 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":259,"threads_started":1}
I20260806 16:06:14.912568 10535 raft_consensus.cc:1261] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f [term 1 FOLLOWER]: Refusing update from remote peer e32b9645dd174e65bb464e5e25079b90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:14.913215 10338 raft_consensus.cc:1261] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81 [term 1 FOLLOWER]: Refusing update from remote peer e32b9645dd174e65bb464e5e25079b90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:14.913518 10647 consensus_queue.cc:1035] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:14.913782 10635 consensus_queue.cc:1035] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:14.915436 10684 tablet_replica.cc:439] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f: TxnStatusTablet state changed. Reason: New leader e32b9645dd174e65bb464e5e25079b90. Latest consensus state: current_term: 1 leader_uuid: "e32b9645dd174e65bb464e5e25079b90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } }
I20260806 16:06:14.915621 10684 tablet_replica.cc:442] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:14.915755 10628 tablet_replica.cc:439] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81: TxnStatusTablet state changed. Reason: New leader e32b9645dd174e65bb464e5e25079b90. Latest consensus state: current_term: 1 leader_uuid: "e32b9645dd174e65bb464e5e25079b90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } }
I20260806 16:06:14.915915 10628 tablet_replica.cc:442] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:14.918689 10680 mvcc.cc:205] Tried to move back new op lower bound from 7315588607636013056 to 7315588607351234560. Current Snapshot: MvccSnapshot[applied={T|T < 7315588607636013056}]
I20260806 16:06:14.918891 10635 tablet_replica.cc:439] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e32b9645dd174e65bb464e5e25079b90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } }
I20260806 16:06:14.919008 10635 tablet_replica.cc:442] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:06:14.920590 10628 tablet_replica.cc:439] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e32b9645dd174e65bb464e5e25079b90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } }
I20260806 16:06:14.920773 10628 tablet_replica.cc:442] T ce1707af045046d98fc8ddde2f7bb803 P 435b228f1d394fd6a981d89b8644bd81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:14.924471 10647 tablet_replica.cc:439] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e32b9645dd174e65bb464e5e25079b90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } }
I20260806 16:06:14.924662 10647 tablet_replica.cc:442] T ce1707af045046d98fc8ddde2f7bb803 P e32b9645dd174e65bb464e5e25079b90: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:06:14.928526 10684 tablet_replica.cc:439] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e32b9645dd174e65bb464e5e25079b90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e32b9645dd174e65bb464e5e25079b90" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34761 } } peers { permanent_uuid: "435b228f1d394fd6a981d89b8644bd81" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34453 } } peers { permanent_uuid: "6de99452d8fc45d396b95fc4bbe0817f" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46429 } } }
I20260806 16:06:14.928733 10684 tablet_replica.cc:442] T ce1707af045046d98fc8ddde2f7bb803 P 6de99452d8fc45d396b95fc4bbe0817f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:14.938915 10697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:14.934505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:34761 (local address 127.8.49.1:53887)
0806 16:06:14.934987 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:14.935000 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:14.935163 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:14.935450 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:14.935453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:14.935460 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:06:14.935707 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:14.935713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.936797 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.936801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:14.937580 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:14.937590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.938044 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.938048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:14.938136 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:14.938198 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:06:14.938517 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:06:14.938525 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 16:06:14.938588 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 16:06:14.938717 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":64,"thread_start_us":203,"threads_started":1}
I20260806 16:06:14.938915 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:14.934593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:53887 (local address 127.8.49.1:34761)
0806 16:06:14.934742 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:14.934746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:14.935091 (+ 345us) server_negotiation.cc:407] Connection header received
0806 16:06:14.935260 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:14.935263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:14.935306 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:14.935444 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:14.936053 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.936660 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.937704 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:14.937927 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:14.938036 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:14.938349 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:14.938426 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:14.938501 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:14.938596 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 16:06:14.938698 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":55}
I20260806 16:06:15.003458 9932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:15.003643 9932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:15.006625 9932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69a14282e8c84da29c7865f0dc1cce06: Sending DeleteTablet for 3 replicas of tablet ef26e69f527943a48fe579da65256dca
I20260806 16:06:15.007761 10508 tablet_service.cc:1514] Processing DeleteTablet for tablet ef26e69f527943a48fe579da65256dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:15 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:44854
I20260806 16:06:15.007788 10309 tablet_service.cc:1514] Processing DeleteTablet for tablet ef26e69f527943a48fe579da65256dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:15 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:40032
I20260806 16:06:15.008220 10701 tablet_replica.cc:330] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f: stopping tablet replica
I20260806 16:06:15.008385 10701 raft_consensus.cc:2229] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:15.008558 10701 raft_consensus.cc:2258] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:15.008562 10132 tablet_service.cc:1514] Processing DeleteTablet for tablet ef26e69f527943a48fe579da65256dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:15 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:58698
I20260806 16:06:15.009479 10701 ts_tablet_manager.cc:1905] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:15.009969 10702 tablet_replica.cc:330] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81: stopping tablet replica
I20260806 16:06:15.010212 10702 raft_consensus.cc:2229] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:15.010464 10702 raft_consensus.cc:2258] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:15.011603 10702 ts_tablet_manager.cc:1905] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:15.013846 10701 ts_tablet_manager.cc:1918] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:15.014047 10701 log.cc:1196] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/ef26e69f527943a48fe579da65256dca
I20260806 16:06:15.014451 10701 ts_tablet_manager.cc:1939] T ef26e69f527943a48fe579da65256dca P 6de99452d8fc45d396b95fc4bbe0817f: Deleting consensus metadata
I20260806 16:06:15.015632 9909 catalog_manager.cc:4841] TS 6de99452d8fc45d396b95fc4bbe0817f (127.8.49.3:46429): tablet ef26e69f527943a48fe579da65256dca (table test-table [id=486391bc1a724733882c45ee5943d4fe]) successfully deleted
I20260806 16:06:15.016165 10707 tablet_replica.cc:330] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90: stopping tablet replica
I20260806 16:06:15.016364 10707 raft_consensus.cc:2229] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:15.016657 10707 raft_consensus.cc:2258] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 06 16:06:15 dist-test-slave-7z32 krb5kdc[9866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:06:15.018157 10707 ts_tablet_manager.cc:1905] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:15.023438 10707 ts_tablet_manager.cc:1918] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:15.023583 10707 log.cc:1196] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-0/wal/wals/ef26e69f527943a48fe579da65256dca
I20260806 16:06:15.023931 10707 ts_tablet_manager.cc:1939] T ef26e69f527943a48fe579da65256dca P e32b9645dd174e65bb464e5e25079b90: Deleting consensus metadata
I20260806 16:06:15.024228 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:15.009111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.8.49.1:34761)
0806 16:06:15.009376 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:15.009383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:15.009519 (+ 136us) server_negotiation.cc:407] Connection header received
0806 16:06:15.009758 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:15.009762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:15.009823 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:15.009879 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:15.010951 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:15.011724 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:15.012893 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:15.013605 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:15.013726 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:15.018019 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:15.018035 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:15.018059 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:15.018093 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:15.021166 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:15.022055 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:15.022078 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:15.022080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:15.022212 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:15.022660 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:15.022664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:15.022666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:15.023104 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:15.023322 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:15.023815 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 16:06:15.024011 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:06:15.024263 10702 ts_tablet_manager.cc:1918] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:15.024766 10702 log.cc:1196] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/ef26e69f527943a48fe579da65256dca
I20260806 16:06:15.024920 9910 catalog_manager.cc:4841] TS e32b9645dd174e65bb464e5e25079b90 (127.8.49.1:34761): tablet ef26e69f527943a48fe579da65256dca (table test-table [id=486391bc1a724733882c45ee5943d4fe]) successfully deleted
I20260806 16:06:15.025209 10702 ts_tablet_manager.cc:1939] T ef26e69f527943a48fe579da65256dca P 435b228f1d394fd6a981d89b8644bd81: Deleting consensus metadata
W20260806 16:06:15.025461 10212 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58734
I20260806 16:06:15.026341 9909 catalog_manager.cc:4841] TS 435b228f1d394fd6a981d89b8644bd81 (127.8.49.2:34453): tablet ef26e69f527943a48fe579da65256dca (table test-table [id=486391bc1a724733882c45ee5943d4fe]) successfully deleted
I20260806 16:06:15.036623 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:15.027858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55996 (local address 127.8.49.62:46497)
0806 16:06:15.027998 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:15.028004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:15.028199 (+ 195us) server_negotiation.cc:407] Connection header received
0806 16:06:15.028361 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:15.028364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:15.028485 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:15.028585 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:15.029294 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:15.029851 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:15.030670 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:15.031047 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:15.031160 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:15.032784 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:15.032793 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:15.032796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:15.032822 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:15.034383 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:15.034983 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:15.034988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:15.034990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:15.035159 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:15.035506 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:15.035511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:15.035513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:15.035797 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:15.036013 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:15.036260 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 16:06:15.036430 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 16:06:15.037374 9932 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55996
I20260806 16:06:15.039238 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 10036
I20260806 16:06:15.062048 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 10230
I20260806 16:06:15.084566 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 10414
I20260806 16:06:15.103036 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 9885
2026-08-06T16:06:15Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4025 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:15 dist-test-slave-7z32 krb5kdc[10712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:15 dist-test-slave-7z32 krb5kdc[10712](info): set up 2 sockets
Aug 06 16:06:15 dist-test-slave-7z32 krb5kdc[10712](info): commencing operation
krb5kdc: starting...
W20260806 16:06:16.536849 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.393s user 0.005s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:16 dist-test-slave-7z32 krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:06:16.716979 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:33693
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:33693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:16.886313 10738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:16.886943 10738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:16.887055 10738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:16.892373 10738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:16.892556 10738 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 16:06:16.892596 10738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:16.892634 10738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:16.892681 10738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:16.892721 10738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:16.901211 10738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:33693
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:33693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:16.903110 10738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:16.904712 10738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:16.919698 10743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:16.919698 10746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:16.919727 10744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:16.920084 10738 server_base.cc:1034] running on GCE node
I20260806 16:06:16.921103 10738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:16.923424 10738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:16.925027 10738 hybrid_clock.cc:648] HybridClock initialized: now 1786032376924929 us; error 107 us; skew 500 ppm
Aug 06 16:06:16 dist-test-slave-7z32 krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:16.930974 10738 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:06:16.932602 10738 webserver.cc:460] Webserver started at http://127.8.49.62:45435/ using document root <none> and password file <none>
I20260806 16:06:16.933494 10738 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:16.933619 10738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:16.933969 10738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:16.937059 10738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "4dba7ac7e6f94be6810be0e354bc972f"
format_stamp: "Formatted at 2026-08-06 16:06:16 on dist-test-slave-7z32"
server_key: "15afb4d2805a7c133f97c637e4d7a051"
server_key_iv: "cc033d85bd299737d202d9e7ba1f76e0"
server_key_version: "encryptionkey@0"
I20260806 16:06:16.937884 10738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "4dba7ac7e6f94be6810be0e354bc972f"
format_stamp: "Formatted at 2026-08-06 16:06:16 on dist-test-slave-7z32"
server_key: "15afb4d2805a7c133f97c637e4d7a051"
server_key_iv: "cc033d85bd299737d202d9e7ba1f76e0"
server_key_version: "encryptionkey@0"
I20260806 16:06:16.943990 10738 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:06:16.948900 10754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:16.951246 10738 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:06:16.951468 10738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "4dba7ac7e6f94be6810be0e354bc972f"
format_stamp: "Formatted at 2026-08-06 16:06:16 on dist-test-slave-7z32"
server_key: "15afb4d2805a7c133f97c637e4d7a051"
server_key_iv: "cc033d85bd299737d202d9e7ba1f76e0"
server_key_version: "encryptionkey@0"
I20260806 16:06:16.951653 10738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:16.973872 10738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:16.979977 10738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:16.980322 10738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:16.995790 10738 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:33693
I20260806 16:06:16.995873 10833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:33693 every 8 connection(s)
I20260806 16:06:16.997706 10738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:17.000624 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 10738
I20260806 16:06:17.000824 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:17.001595 8388 external_mini_cluster.cc:1428] Setting key 3f859ef8aa70563915bdec1dcefd8a7b
I20260806 16:06:17.002362 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:17.013913 10834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f: Bootstrap starting.
I20260806 16:06:17.018157 10834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f: Neither blocks nor log segments found. Creating new log.
Aug 06 16:06:17 dist-test-slave-7z32 krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:17.019237 10834 log.cc:824] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:17.023592 10834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f: No bootstrap required, opened a new log
I20260806 16:06:17.031503 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.004980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59136 (local address 127.8.49.62:33693)
0806 16:06:17.005805 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:17.005829 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:17.005887 (+ 58us) server_negotiation.cc:407] Connection header received
0806 16:06:17.006769 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:17.006796 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:17.007437 (+ 641us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:17.007906 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:17.009834 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.011646 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.013755 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.015511 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.015783 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:17.020239 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:17.020251 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:17.020275 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:17.020324 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:17.024896 (+ 4572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.025958 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.025966 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.025976 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.026163 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.027091 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.027097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.027099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.027895 (+ 796us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:17.028164 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:17.029660 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0806 16:06:17.030121 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":260,"threads_started":1}
I20260806 16:06:17.031231 10834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dba7ac7e6f94be6810be0e354bc972f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33693 } }
I20260806 16:06:17.031692 10834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:17.031767 10834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dba7ac7e6f94be6810be0e354bc972f, State: Initialized, Role: FOLLOWER
I20260806 16:06:17.032824 10834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dba7ac7e6f94be6810be0e354bc972f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33693 } }
I20260806 16:06:17.034750 10834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:17.035048 10834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:17.035238 10834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:17.036810 10834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dba7ac7e6f94be6810be0e354bc972f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33693 } }
I20260806 16:06:17.037379 10834 leader_election.cc:302] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dba7ac7e6f94be6810be0e354bc972f; no voters:
I20260806 16:06:17.037914 10834 leader_election.cc:288] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:17.038201 10841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:17.038726 10841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [term 1 LEADER]: Becoming Leader. State: Replica: 4dba7ac7e6f94be6810be0e354bc972f, State: Running, Role: LEADER
I20260806 16:06:17.039266 10841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dba7ac7e6f94be6810be0e354bc972f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33693 } }
I20260806 16:06:17.040038 10834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:17.041770 10842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4dba7ac7e6f94be6810be0e354bc972f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dba7ac7e6f94be6810be0e354bc972f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33693 } } }
I20260806 16:06:17.042068 10842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:17.042526 10845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dba7ac7e6f94be6810be0e354bc972f. Latest consensus state: current_term: 1 leader_uuid: "4dba7ac7e6f94be6810be0e354bc972f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dba7ac7e6f94be6810be0e354bc972f" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33693 } } }
I20260806 16:06:17.042733 10854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:17.042778 10845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:17.048266 10854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:17.057163 10854 catalog_manager.cc:1347] Generated new cluster ID: f1c6a42865d4486eaaff6028e5e1da9c
I20260806 16:06:17.057296 10854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:17.069996 10854 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:17.071851 10854 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:17.080724 10854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f: Generated new TSK 0
I20260806 16:06:17.081975 10854 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:06:17.131507 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:33693
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:17.299741 10869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:17.300194 10869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:17.300259 10869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:17.305429 10869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:17.305540 10869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:17.305593 10869 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 16:06:17.305608 10869 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:06:17.305645 10869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:17.312674 10869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.49.62:33693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:17.314347 10869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:17.315932 10869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:17.329094 10876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:17.329082 10878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:17.329540 10869 server_base.cc:1034] running on GCE node
W20260806 16:06:17.329082 10880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:17.330139 10869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:17.331000 10869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:17.332259 10869 hybrid_clock.cc:648] HybridClock initialized: now 1786032377332230 us; error 49 us; skew 500 ppm
Aug 06 16:06:17 dist-test-slave-7z32 krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032377, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:17.337177 10869 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:06:17.338742 10869 webserver.cc:460] Webserver started at http://127.8.49.1:40999/ using document root <none> and password file <none>
I20260806 16:06:17.339612 10869 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:17.339740 10869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:17.340080 10869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:17.343220 10869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "047cfb4f2a9e4d088b46199e48bc2474"
format_stamp: "Formatted at 2026-08-06 16:06:17 on dist-test-slave-7z32"
server_key: "3c61e61fd13491e359069036040ea790"
server_key_iv: "f79a18409b4097edff628abeb3a25787"
server_key_version: "encryptionkey@0"
I20260806 16:06:17.344134 10869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "047cfb4f2a9e4d088b46199e48bc2474"
format_stamp: "Formatted at 2026-08-06 16:06:17 on dist-test-slave-7z32"
server_key: "3c61e61fd13491e359069036040ea790"
server_key_iv: "f79a18409b4097edff628abeb3a25787"
server_key_version: "encryptionkey@0"
I20260806 16:06:17.350719 10869 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260806 16:06:17.354230 10888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:17.356359 10869 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:17.356585 10869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "047cfb4f2a9e4d088b46199e48bc2474"
format_stamp: "Formatted at 2026-08-06 16:06:17 on dist-test-slave-7z32"
server_key: "3c61e61fd13491e359069036040ea790"
server_key_iv: "f79a18409b4097edff628abeb3a25787"
server_key_version: "encryptionkey@0"
I20260806 16:06:17.356835 10869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:17.373605 10869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:17.378137 10869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:17.378455 10869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:17.380509 10869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:17.380630 10869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:17.380728 10869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:17.380782 10869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:06:17 dist-test-slave-7z32 krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032377, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:17.396270 10869 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:38821
I20260806 16:06:17.396365 11049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:38821 every 8 connection(s)
I20260806 16:06:17.397814 10869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:17.401798 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.383322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:60115 (local address 127.8.49.62:33693)
0806 16:06:17.383515 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:17.383521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:17.385207 (+ 1686us) server_negotiation.cc:407] Connection header received
0806 16:06:17.386864 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:17.386869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:17.386959 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:17.387089 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:17.389024 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.389886 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.391826 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.392263 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.392410 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:17.396630 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:17.396637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:17.396640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:17.396687 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:17.399173 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.399933 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.399937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.399940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.400040 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.400458 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.400464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.400466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.400714 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:17.400840 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:17.401432 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 16:06:17.401601 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 16:06:17.402793 10907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.384076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:33693 (local address 127.8.49.1:60115)
0806 16:06:17.384948 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:17.385021 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:17.386518 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:17.387363 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:17.387372 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:17.387887 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:17.388785 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:17.388816 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.390059 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.390063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:17.391661 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:17.391675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.392488 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.392492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:17.392737 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:17.393784 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:17.393808 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:17.396277 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:17.399389 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:17.399407 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:17.399438 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:17.399787 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:17.400157 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:17.400160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:17.400163 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:17.400331 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:17.400841 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:17.400850 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:17.401158 (+ 308us) client_negotiation.cc:769] Sending connection context
0806 16:06:17.401385 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 16:06:17.401802 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":277,"threads_started":1}
I20260806 16:06:17.405020 11050 heartbeater.cc:347] Connected to a master server at 127.8.49.62:33693
I20260806 16:06:17.405273 11050 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:17.405489 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 10869
I20260806 16:06:17.405675 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:17.405931 11050 heartbeater.cc:511] Master 127.8.49.62:33693 requested a full tablet report, sending...
I20260806 16:06:17.406085 8388 external_mini_cluster.cc:1428] Setting key 164bcc35fb1ebbc9732cba1c2e248dba
I20260806 16:06:17.407871 10776 ts_manager.cc:194] Registered new tserver with Master: 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821)
I20260806 16:06:17.409190 10776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:60115
I20260806 16:06:17.421926 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.409298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:32929 (local address 127.8.49.62:33693)
0806 16:06:17.409490 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:17.409499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:17.410317 (+ 818us) server_negotiation.cc:407] Connection header received
0806 16:06:17.410511 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:17.410515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:17.410589 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:17.410700 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:17.412317 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.413679 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.414743 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.415252 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.415426 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:17.417198 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:17.417202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:17.417204 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:17.417237 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:17.419152 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.420177 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.420181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.420184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.420289 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.420771 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.420773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.420775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.421008 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:17.421109 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:17.421624 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 16:06:17.421775 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:06:17.422122 11057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.409622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:33693 (local address 127.8.49.1:32929)
0806 16:06:17.410199 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:17.410221 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:17.410381 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:17.410772 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:17.410778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:17.411391 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:17.411939 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:17.411957 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.413878 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.413881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:17.414574 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:17.414581 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.415476 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.415480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:17.415579 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:17.416476 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:17.416491 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:17.417010 (+ 519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:17.419321 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:17.419328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:17.419332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:17.419979 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:17.420459 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:17.420464 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:17.420466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:17.420622 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:17.421141 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:17.421146 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:17.421342 (+ 196us) client_negotiation.cc:769] Sending connection context
0806 16:06:17.421501 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 16:06:17.421755 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":179,"threads_started":1}
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:06:17.447603 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:33693
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:17.595167 11062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:17.595503 11062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:17.595561 11062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:17.600632 11062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:17.600736 11062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:17.600783 11062 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 16:06:17.600800 11062 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:06:17.600874 11062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:17.607071 11062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.49.62:33693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:17.608423 11062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:17.609634 11062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:17.621910 11073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:17.622159 11062 server_base.cc:1034] running on GCE node
W20260806 16:06:17.621866 11070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:17.621945 11071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:17.623369 11062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:17.624432 11062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:17.625787 11062 hybrid_clock.cc:648] HybridClock initialized: now 1786032377625757 us; error 53 us; skew 500 ppm
Aug 06 16:06:17 dist-test-slave-7z32 krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032377, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:17.630355 11062 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:06:17.631924 11062 webserver.cc:460] Webserver started at http://127.8.49.2:46393/ using document root <none> and password file <none>
I20260806 16:06:17.632757 11062 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:17.632843 11062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:17.633091 11062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:17.636360 11062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "598defb7de294eaa8020b041a87c5a1a"
format_stamp: "Formatted at 2026-08-06 16:06:17 on dist-test-slave-7z32"
server_key: "e105006955e3aafa2d050f07ef97d646"
server_key_iv: "d3a27584cc57cf3a7896742c96806265"
server_key_version: "encryptionkey@0"
I20260806 16:06:17.637228 11062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "598defb7de294eaa8020b041a87c5a1a"
format_stamp: "Formatted at 2026-08-06 16:06:17 on dist-test-slave-7z32"
server_key: "e105006955e3aafa2d050f07ef97d646"
server_key_iv: "d3a27584cc57cf3a7896742c96806265"
server_key_version: "encryptionkey@0"
I20260806 16:06:17.644366 11062 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 16:06:17.648747 11084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:17.651084 11062 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260806 16:06:17.651322 11062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "598defb7de294eaa8020b041a87c5a1a"
format_stamp: "Formatted at 2026-08-06 16:06:17 on dist-test-slave-7z32"
server_key: "e105006955e3aafa2d050f07ef97d646"
server_key_iv: "d3a27584cc57cf3a7896742c96806265"
server_key_version: "encryptionkey@0"
I20260806 16:06:17.651508 11062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:17.669075 11062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:17.674093 11062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:17.674368 11062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:17.676614 11062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:17.676692 11062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:17.676749 11062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:17.676853 11062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:06:17 dist-test-slave-7z32 krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032377, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:17.693325 11062 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:41799
I20260806 16:06:17.695119 11062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:17.696376 11242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:41799 every 8 connection(s)
I20260806 16:06:17.697265 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.678070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:50735 (local address 127.8.49.62:33693)
0806 16:06:17.678346 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:17.678352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:17.679508 (+ 1156us) server_negotiation.cc:407] Connection header received
0806 16:06:17.680641 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:17.680645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:17.680724 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:17.680824 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:17.682722 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.683453 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.684990 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.685408 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.685576 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:17.691235 (+ 5659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:17.691240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:17.691244 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:17.691285 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:17.693576 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.694461 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.694471 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.694476 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.694658 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.695319 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.695324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.695326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.695701 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:17.695868 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:17.696829 (+ 961us) server_negotiation.cc:299] Negotiation successful
0806 16:06:17.697039 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 16:06:17.698964 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.678504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:33693 (local address 127.8.49.2:50735)
0806 16:06:17.679178 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:17.679244 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:17.680337 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:17.681194 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:17.681202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:17.681897 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:17.682526 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:17.682544 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.683618 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.683623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:17.684811 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:17.684824 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.685714 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.685724 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:17.686156 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:17.687308 (+ 1152us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:17.687340 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:17.690908 (+ 3568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:17.693785 (+ 2877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:17.693796 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:17.693824 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:17.694258 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:17.694891 (+ 633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:17.694895 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:17.694897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:17.695112 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:17.695949 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:17.695960 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:17.696424 (+ 464us) client_negotiation.cc:769] Sending connection context
0806 16:06:17.696750 (+ 326us) client_negotiation.cc:240] Negotiation successful
0806 16:06:17.697231 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":181,"threads_started":1}
I20260806 16:06:17.700153 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 11062
I20260806 16:06:17.700320 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:17.700719 8388 external_mini_cluster.cc:1428] Setting key cb2f2a437fc980d0072f252dc5bdfc6c
I20260806 16:06:17.701643 11243 heartbeater.cc:347] Connected to a master server at 127.8.49.62:33693
I20260806 16:06:17.701934 11243 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:17.703172 11243 heartbeater.cc:511] Master 127.8.49.62:33693 requested a full tablet report, sending...
I20260806 16:06:17.705130 10775 ts_manager.cc:194] Registered new tserver with Master: 598defb7de294eaa8020b041a87c5a1a (127.8.49.2:41799)
I20260806 16:06:17.706430 10775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:50735
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:06:17.719643 11253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.705547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:33693 (local address 127.8.49.2:35235)
0806 16:06:17.705895 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:17.705910 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:17.706125 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:17.706636 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:17.706639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:17.706985 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:17.707163 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:17.707173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.708510 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.708516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:17.709781 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:17.709803 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.710514 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.710522 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:17.710687 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:17.711475 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:17.711511 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:17.712195 (+ 684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:17.716424 (+ 4229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:17.716430 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:17.716433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:17.716829 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:17.717662 (+ 833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:17.717668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:17.717670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:17.717862 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:17.718634 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:17.718638 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:17.718778 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 16:06:17.718953 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 16:06:17.719358 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":44,"thread_start_us":145,"threads_started":1}
I20260806 16:06:17.719615 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.705507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:35235 (local address 127.8.49.62:33693)
0806 16:06:17.705686 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:17.705694 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:17.706005 (+ 311us) server_negotiation.cc:407] Connection header received
0806 16:06:17.706262 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:17.706265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:17.706329 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:17.706440 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:17.707368 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.708258 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.709999 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.710314 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.710442 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:17.712460 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:17.712464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:17.712467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:17.712511 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:17.715958 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.717000 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.717005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.717006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.717134 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.718096 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.718105 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.718107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.718480 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:17.718648 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:17.719191 (+ 543us) server_negotiation.cc:299] Negotiation successful
0806 16:06:17.719358 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:06:17.748733 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:33693
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:17.892279 11256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:17.892697 11256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:17.892793 11256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:17.898114 11256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:17.898278 11256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:17.898378 11256 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 16:06:17.898413 11256 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:06:17.898576 11256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:17.905169 11256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.49.62:33693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:17.906548 11256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:17.907837 11256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:17.920959 11263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:17.920822 11265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:17.920815 11262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:17.921348 11256 server_base.cc:1034] running on GCE node
I20260806 16:06:17.922196 11256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:17.923280 11256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:17.924623 11256 hybrid_clock.cc:648] HybridClock initialized: now 1786032377924563 us; error 69 us; skew 500 ppm
Aug 06 16:06:17 dist-test-slave-7z32 krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032377, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:17.929534 11256 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:06:17.931229 11256 webserver.cc:460] Webserver started at http://127.8.49.3:38971/ using document root <none> and password file <none>
I20260806 16:06:17.932102 11256 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:17.932192 11256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:17.932495 11256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:17.935686 11256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "98ea9449706a41d89b32b412374c0df8"
format_stamp: "Formatted at 2026-08-06 16:06:17 on dist-test-slave-7z32"
server_key: "cfa6ca6935343434123cb8411f37b7c4"
server_key_iv: "326e8769ba1a9cd3d9c1e74eb5f208ea"
server_key_version: "encryptionkey@0"
I20260806 16:06:17.936506 11256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "98ea9449706a41d89b32b412374c0df8"
format_stamp: "Formatted at 2026-08-06 16:06:17 on dist-test-slave-7z32"
server_key: "cfa6ca6935343434123cb8411f37b7c4"
server_key_iv: "326e8769ba1a9cd3d9c1e74eb5f208ea"
server_key_version: "encryptionkey@0"
I20260806 16:06:17.943514 11256 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 16:06:17.947683 11275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:17.949793 11256 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:17.950069 11256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "98ea9449706a41d89b32b412374c0df8"
format_stamp: "Formatted at 2026-08-06 16:06:17 on dist-test-slave-7z32"
server_key: "cfa6ca6935343434123cb8411f37b7c4"
server_key_iv: "326e8769ba1a9cd3d9c1e74eb5f208ea"
server_key_version: "encryptionkey@0"
I20260806 16:06:17.950234 11256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:17.965025 11256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:17.969694 11256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:17.969929 11256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:17.972257 11256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:17.972342 11256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:17.972397 11256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:17.972414 11256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:06:17 dist-test-slave-7z32 krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032377, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:17.987831 11256 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:44741
I20260806 16:06:17.987900 11420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:44741 every 8 connection(s)
I20260806 16:06:17.989351 11256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:17.989938 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.973523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:59525 (local address 127.8.49.62:33693)
0806 16:06:17.973753 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:17.973759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:17.974732 (+ 973us) server_negotiation.cc:407] Connection header received
0806 16:06:17.975938 (+ 1206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:17.975943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:17.976023 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:17.976123 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:17.977705 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.978271 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.979901 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.980136 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.980257 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:17.984133 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:17.984140 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:17.984146 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:17.984206 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:17.986679 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.987376 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.987381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.987384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.987507 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:17.988156 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:17.988160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:17.988161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:17.988380 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:17.988475 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:17.989588 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0806 16:06:17.989773 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 16:06:17.991302 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 11256
I20260806 16:06:17.991475 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:17.991607 11287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.973906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:33693 (local address 127.8.49.3:59525)
0806 16:06:17.974518 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:17.974576 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:17.975644 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:17.976338 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:17.976348 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:17.976800 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:17.977522 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:17.977541 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.978452 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.978456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:17.979745 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:17.979757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.980326 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.980331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:17.980573 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:17.981369 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:17.981389 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:17.983819 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:17.986861 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:17.986869 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:17.986891 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:17.987242 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:17.987661 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:17.987669 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:17.987675 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:17.987992 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:17.988514 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:17.988522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:17.988964 (+ 442us) client_negotiation.cc:769] Sending connection context
0806 16:06:17.989220 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 16:06:17.989874 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":178,"threads_started":1}
I20260806 16:06:17.991950 8388 external_mini_cluster.cc:1428] Setting key e58ce0431f1e1e1e3816926b351d9dee
I20260806 16:06:17.993484 11421 heartbeater.cc:347] Connected to a master server at 127.8.49.62:33693
I20260806 16:06:17.993753 11421 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:17.994531 11421 heartbeater.cc:511] Master 127.8.49.62:33693 requested a full tablet report, sending...
I20260806 16:06:18.002942 10775 ts_manager.cc:194] Registered new tserver with Master: 98ea9449706a41d89b32b412374c0df8 (127.8.49.3:44741)
I20260806 16:06:18.003911 10775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:59525
I20260806 16:06:18.006984 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:06:18.009877 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.997110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:33693 (local address 127.8.49.3:59311)
0806 16:06:17.997520 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:17.997537 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:17.997669 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:17.997955 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:17.997957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:17.998223 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:17.998404 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:17.998410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.999757 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:17.999763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.000739 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.000760 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.001693 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.001697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.001815 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.002656 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:18.002675 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:18.003402 (+ 727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:18.006684 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:18.006688 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:18.006691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:18.007077 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:18.007544 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:18.007673 (+ 129us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:18.007675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:18.007803 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:18.008800 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:18.008804 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:18.008951 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.009286 (+ 335us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.009565 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":151,"threads_started":1}
I20260806 16:06:18.009863 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:17.997202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:59311 (local address 127.8.49.62:33693)
0806 16:06:17.997351 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:17.997356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:17.997639 (+ 283us) server_negotiation.cc:407] Connection header received
0806 16:06:17.997772 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:17.997775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:17.997830 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:17.997942 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:17.998533 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:17.999418 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.001001 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.001376 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.001496 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.003625 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:18.003630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:18.003633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:18.003672 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:18.006423 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:18.007234 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:18.007239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:18.007242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:18.007379 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:18.007953 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:18.007957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:18.007959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:18.008492 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:18.008657 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.009373 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.009560 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 16:06:18 dist-test-slave-7z32 krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 16:06:18 dist-test-slave-7z32 krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:18.044121 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.031145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59144 (local address 127.8.49.62:33693)
0806 16:06:18.031376 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.031381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.031420 (+ 39us) server_negotiation.cc:407] Connection header received
0806 16:06:18.031493 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.031495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.031541 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.031641 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:18.032831 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.033771 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.035526 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.035860 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.035997 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.039521 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:18.039527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:18.039529 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:18.039581 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:18.041547 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:18.042320 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:18.042324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:18.042325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:18.042435 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:18.042949 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:18.042952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:18.042954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:18.043251 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:18.043401 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.043800 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.043955 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 16:06:18.049439 10775 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:18.053574 10775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:18.073875 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.065837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.8.49.3:44741)
0806 16:06:18.066409 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.066423 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.067667 (+ 1244us) server_negotiation.cc:407] Connection header received
0806 16:06:18.067798 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.067806 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.068130 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.068358 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:18.068890 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.069804 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.072064 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.072867 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.073029 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.073284 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.073494 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.073637 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":182,"threads_started":1}
I20260806 16:06:18.073858 11441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.065708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:44741 (local address 127.0.0.1:33928)
0806 16:06:18.067259 (+ 1551us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:18.067279 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:18.067454 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:18.068346 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:18.068354 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:18.068396 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:18.068718 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.068731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.069954 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.069962 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.071881 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.071897 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.073037 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.073041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.073204 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.073260 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.073448 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.073652 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1449,"mutex_wait_us":222,"spinlock_wait_cycles":7168,"thread_start_us":104,"threads_started":1}
I20260806 16:06:18.074302 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.066051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45738 (local address 127.8.49.2:41799)
0806 16:06:18.066439 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.066451 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.067491 (+ 1040us) server_negotiation.cc:407] Connection header received
0806 16:06:18.067703 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.067713 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.067987 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.068157 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:18.068815 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.070105 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.072054 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.073014 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.073131 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.073322 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.073978 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.074113 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":100,"threads_started":1}
I20260806 16:06:18.074685 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.067040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:38821 (local address 127.0.0.1:50826)
0806 16:06:18.067797 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:18.067812 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:18.067917 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:18.068604 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:18.068608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:18.068630 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:18.068752 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.068761 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.070807 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.070858 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.071953 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.071994 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.073391 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.073396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.073600 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.073687 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.073763 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.074521 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":747,"thread_start_us":144,"threads_started":1}
I20260806 16:06:18.074915 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.065708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:41799 (local address 127.0.0.1:45738)
0806 16:06:18.066742 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:18.066786 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:18.067045 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:18.068344 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:18.068350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:18.068398 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:18.068667 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.068676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.070264 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.070268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.071881 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.071897 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.073391 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.073396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.073621 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.073681 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.073763 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.073889 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":161,"spinlock_wait_cycles":6656,"thread_start_us":259,"threads_started":1}
I20260806 16:06:18.075156 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.067182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50826 (local address 127.8.49.1:38821)
0806 16:06:18.067639 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.067646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.068018 (+ 372us) server_negotiation.cc:407] Connection header received
0806 16:06:18.068112 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.068117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.068327 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.068481 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:18.068885 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.070105 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.072153 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.073014 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.073131 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.073310 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.073875 (+ 565us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.074616 (+ 741us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":109,"threads_started":1}
I20260806 16:06:18.078652 11149 tablet_service.cc:1467] Processing CreateTablet for tablet b882c5a8f8ac4b1387c753544d08a276 (DEFAULT_TABLE table=test-table [id=e809b2bf57f54dd5aa93baff7fff8b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:18.078626 10954 tablet_service.cc:1467] Processing CreateTablet for tablet b882c5a8f8ac4b1387c753544d08a276 (DEFAULT_TABLE table=test-table [id=e809b2bf57f54dd5aa93baff7fff8b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:18.079067 11336 tablet_service.cc:1467] Processing CreateTablet for tablet b882c5a8f8ac4b1387c753544d08a276 (DEFAULT_TABLE table=test-table [id=e809b2bf57f54dd5aa93baff7fff8b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:18.080058 10954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b882c5a8f8ac4b1387c753544d08a276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:18.080058 11149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b882c5a8f8ac4b1387c753544d08a276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:18.080487 11336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b882c5a8f8ac4b1387c753544d08a276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:18.088919 11453 tablet_bootstrap.cc:492] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: Bootstrap starting.
I20260806 16:06:18.089589 11450 tablet_bootstrap.cc:492] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: Bootstrap starting.
I20260806 16:06:18.090615 11455 tablet_bootstrap.cc:492] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: Bootstrap starting.
I20260806 16:06:18.093165 11455 tablet_bootstrap.cc:654] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:18.093535 11453 tablet_bootstrap.cc:654] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:18.093535 11450 tablet_bootstrap.cc:654] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:18.094229 11455 log.cc:824] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:18.094642 11453 log.cc:824] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:18.094642 11450 log.cc:824] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:18.096990 11455 tablet_bootstrap.cc:492] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: No bootstrap required, opened a new log
I20260806 16:06:18.097316 11455 ts_tablet_manager.cc:1397] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:06:18.097307 11450 tablet_bootstrap.cc:492] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: No bootstrap required, opened a new log
I20260806 16:06:18.097307 11453 tablet_bootstrap.cc:492] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: No bootstrap required, opened a new log
I20260806 16:06:18.097576 11450 ts_tablet_manager.cc:1397] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 16:06:18.097575 11453 ts_tablet_manager.cc:1397] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 16:06:18.102033 11455 raft_consensus.cc:348] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } }
I20260806 16:06:18.102491 11455 raft_consensus.cc:374] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:18.102574 11455 raft_consensus.cc:729] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ea9449706a41d89b32b412374c0df8, State: Initialized, Role: FOLLOWER
I20260806 16:06:18.102468 11450 raft_consensus.cc:348] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } }
I20260806 16:06:18.102386 11453 raft_consensus.cc:348] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } }
I20260806 16:06:18.102756 11450 raft_consensus.cc:374] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:18.102756 11453 raft_consensus.cc:374] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:18.102860 11450 raft_consensus.cc:729] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047cfb4f2a9e4d088b46199e48bc2474, State: Initialized, Role: FOLLOWER
I20260806 16:06:18.102878 11453 raft_consensus.cc:729] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598defb7de294eaa8020b041a87c5a1a, State: Initialized, Role: FOLLOWER
I20260806 16:06:18.103379 11455 consensus_queue.cc:260] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } }
I20260806 16:06:18.103479 11453 consensus_queue.cc:260] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } }
I20260806 16:06:18.103479 11450 consensus_queue.cc:260] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } }
I20260806 16:06:18.104516 11050 heartbeater.cc:503] Master 127.8.49.62:33693 was elected leader, sending a full tablet report...
I20260806 16:06:18.104780 11453 ts_tablet_manager.cc:1428] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260806 16:06:18.104780 11450 ts_tablet_manager.cc:1428] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:06:18.104947 11455 ts_tablet_manager.cc:1428] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260806 16:06:18.105257 11243 heartbeater.cc:503] Master 127.8.49.62:33693 was elected leader, sending a full tablet report...
I20260806 16:06:18.105326 11421 heartbeater.cc:503] Master 127.8.49.62:33693 was elected leader, sending a full tablet report...
W20260806 16:06:18.150353 11051 tablet.cc:2369] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:06:18.197196 11246 tablet.cc:2369] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:06:18.241456 11422 tablet.cc:2369] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:18.343815 11461 raft_consensus.cc:482] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:18.344292 11461 raft_consensus.cc:504] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } }
I20260806 16:06:18.346093 11461 leader_election.cc:288] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98ea9449706a41d89b32b412374c0df8 (127.8.49.3:44741), 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821)
I20260806 16:06:18.350716 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.346719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:36807 (local address 127.8.49.3:44741)
0806 16:06:18.346926 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.346930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.347027 (+ 97us) server_negotiation.cc:407] Connection header received
0806 16:06:18.347256 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.347264 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.347332 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.347466 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:18.347867 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.348423 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.349642 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.350069 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.350197 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.350287 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.350478 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.350559 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:06:18.350732 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.346450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:44741 (local address 127.8.49.2:36807)
0806 16:06:18.346894 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:18.346917 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:18.347088 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:18.347508 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:18.347514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:18.347548 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:18.347736 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.347744 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.348567 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.348571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.349497 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.349510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.350211 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.350215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.350334 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.350360 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.350458 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.350559 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":171,"threads_started":1}
I20260806 16:06:18.351372 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.347052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:38821 (local address 127.8.49.2:52747)
0806 16:06:18.347442 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:18.347459 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:18.347577 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:18.347893 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:18.347897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:18.347916 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:18.348083 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.348090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.349004 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.349010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.349926 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.349936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.350750 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.350754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.350910 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.350947 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.351083 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.351186 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"spinlock_wait_cycles":4352,"thread_start_us":101,"threads_started":1}
I20260806 16:06:18.351361 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.347319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:52747 (local address 127.8.49.1:38821)
0806 16:06:18.347551 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.347558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.347578 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:06:18.347696 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.347701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.347772 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.347878 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:18.348229 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.348861 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.350084 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.350590 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.350736 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.350822 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.351108 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.351218 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 16:06:18.351595 11351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b882c5a8f8ac4b1387c753544d08a276" candidate_uuid: "598defb7de294eaa8020b041a87c5a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ea9449706a41d89b32b412374c0df8" is_pre_election: true
I20260806 16:06:18.352054 11351 raft_consensus.cc:2454] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 598defb7de294eaa8020b041a87c5a1a in term 0.
I20260806 16:06:18.352260 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b882c5a8f8ac4b1387c753544d08a276" candidate_uuid: "598defb7de294eaa8020b041a87c5a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047cfb4f2a9e4d088b46199e48bc2474" is_pre_election: true
I20260806 16:06:18.352571 10977 raft_consensus.cc:2454] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 598defb7de294eaa8020b041a87c5a1a in term 0.
I20260806 16:06:18.352787 11086 leader_election.cc:302] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598defb7de294eaa8020b041a87c5a1a, 98ea9449706a41d89b32b412374c0df8; no voters:
I20260806 16:06:18.353171 11461 raft_consensus.cc:2781] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:18.353430 11461 raft_consensus.cc:482] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:18.353488 11461 raft_consensus.cc:3037] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:18.355057 11461 raft_consensus.cc:504] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } }
I20260806 16:06:18.355509 11461 leader_election.cc:288] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [CANDIDATE]: Term 1 election: Requested vote from peers 98ea9449706a41d89b32b412374c0df8 (127.8.49.3:44741), 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821)
I20260806 16:06:18.356185 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b882c5a8f8ac4b1387c753544d08a276" candidate_uuid: "598defb7de294eaa8020b041a87c5a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047cfb4f2a9e4d088b46199e48bc2474"
I20260806 16:06:18.356185 11351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b882c5a8f8ac4b1387c753544d08a276" candidate_uuid: "598defb7de294eaa8020b041a87c5a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ea9449706a41d89b32b412374c0df8"
I20260806 16:06:18.356355 11351 raft_consensus.cc:3037] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:18.356355 10977 raft_consensus.cc:3037] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:18.358103 10977 raft_consensus.cc:2454] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 598defb7de294eaa8020b041a87c5a1a in term 1.
I20260806 16:06:18.358686 11086 leader_election.cc:302] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047cfb4f2a9e4d088b46199e48bc2474, 598defb7de294eaa8020b041a87c5a1a; no voters:
I20260806 16:06:18.358803 11351 raft_consensus.cc:2454] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 598defb7de294eaa8020b041a87c5a1a in term 1.
I20260806 16:06:18.359031 11461 raft_consensus.cc:2781] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:18.359294 11461 raft_consensus.cc:686] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 1 LEADER]: Becoming Leader. State: Replica: 598defb7de294eaa8020b041a87c5a1a, State: Running, Role: LEADER
I20260806 16:06:18.359723 11461 consensus_queue.cc:237] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } }
I20260806 16:06:18.364420 10775 catalog_manager.cc:5495] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 598defb7de294eaa8020b041a87c5a1a (127.8.49.2). New cstate: current_term: 1 leader_uuid: "598defb7de294eaa8020b041a87c5a1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } health_report { overall_health: UNKNOWN } } }
Aug 06 16:06:18 dist-test-slave-7z32 krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:18.421309 11474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.405405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:33693 (local address 127.0.0.1:59148)
0806 16:06:18.405871 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:18.405886 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:18.406078 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:18.406382 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:18.406385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:18.407059 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:18.407229 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.407244 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.408677 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.408682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.409936 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.409957 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.411034 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.411040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.411186 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.412224 (+ 1038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:18.412239 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:18.414792 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:18.418916 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:18.418941 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:18.418960 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:18.419464 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:18.419860 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:18.419863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:18.419864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:18.419961 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:18.420568 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:18.420575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:18.420753 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.420869 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.421150 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":212,"threads_started":1}
I20260806 16:06:18.421598 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.405545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.8.49.62:33693)
0806 16:06:18.405738 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.405746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.406013 (+ 267us) server_negotiation.cc:407] Connection header received
0806 16:06:18.406187 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.406191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.406244 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.406344 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:18.407463 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.408467 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.410185 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.410775 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.411013 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.415180 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:18.415186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:18.415190 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:18.415353 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:18.418687 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:18.419646 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:18.419649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:18.419651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:18.419762 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:18.420091 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:18.420095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:18.420096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:18.420365 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:18.420465 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.421132 (+ 667us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.421306 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 16:06:18.427284 10775 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:59148:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 16:06:18.428159 10775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:18.436367 11149 tablet_service.cc:1467] Processing CreateTablet for tablet 14bca24762f3409d965212899e1534a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=05acd25ab3c14bbfbf3968b3b16c4162]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:06:18.436362 10954 tablet_service.cc:1467] Processing CreateTablet for tablet 14bca24762f3409d965212899e1534a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=05acd25ab3c14bbfbf3968b3b16c4162]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:06:18.437281 11149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14bca24762f3409d965212899e1534a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:18.437458 10954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14bca24762f3409d965212899e1534a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:18.441099 11453 tablet_bootstrap.cc:492] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a: Bootstrap starting.
I20260806 16:06:18.441731 11450 tablet_bootstrap.cc:492] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474: Bootstrap starting.
I20260806 16:06:18.441699 11336 tablet_service.cc:1467] Processing CreateTablet for tablet 14bca24762f3409d965212899e1534a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=05acd25ab3c14bbfbf3968b3b16c4162]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:06:18.442708 11336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14bca24762f3409d965212899e1534a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:18.442992 11453 tablet_bootstrap.cc:654] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:18.443228 11450 tablet_bootstrap.cc:654] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:18.445030 11453 tablet_bootstrap.cc:492] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a: No bootstrap required, opened a new log
I20260806 16:06:18.445302 11453 ts_tablet_manager.cc:1397] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260806 16:06:18.445353 11450 tablet_bootstrap.cc:492] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474: No bootstrap required, opened a new log
I20260806 16:06:18.445441 11450 ts_tablet_manager.cc:1397] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:18.445996 11453 raft_consensus.cc:348] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.445996 11450 raft_consensus.cc:348] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.446161 11453 raft_consensus.cc:374] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:18.446200 11453 raft_consensus.cc:729] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598defb7de294eaa8020b041a87c5a1a, State: Initialized, Role: FOLLOWER
I20260806 16:06:18.446373 11453 consensus_queue.cc:260] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.446448 11455 tablet_bootstrap.cc:492] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: Bootstrap starting.
I20260806 16:06:18.446754 11453 ts_tablet_manager.cc:1428] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260806 16:06:18.446161 11450 raft_consensus.cc:374] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:18.446918 11450 raft_consensus.cc:729] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047cfb4f2a9e4d088b46199e48bc2474, State: Initialized, Role: FOLLOWER
I20260806 16:06:18.447103 11450 consensus_queue.cc:260] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.447321 11461 tablet_replica.cc:439] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } }
I20260806 16:06:18.447469 11450 ts_tablet_manager.cc:1428] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 16:06:18.447502 11461 tablet_replica.cc:442] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:18.447795 11455 tablet_bootstrap.cc:654] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:18.448742 11460 tablet_replica.cc:439] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } }
I20260806 16:06:18.448944 11460 tablet_replica.cc:442] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:18.449720 11455 tablet_bootstrap.cc:492] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: No bootstrap required, opened a new log
I20260806 16:06:18.449816 11455 ts_tablet_manager.cc:1397] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:06:18.450338 11455 raft_consensus.cc:348] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.450482 11455 raft_consensus.cc:374] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:18.450537 11455 raft_consensus.cc:729] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ea9449706a41d89b32b412374c0df8, State: Initialized, Role: FOLLOWER
I20260806 16:06:18.450733 11455 consensus_queue.cc:260] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.451149 11455 ts_tablet_manager.cc:1428] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:06:18.451261 11462 tablet_replica.cc:439] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } }
I20260806 16:06:18.451550 11462 tablet_replica.cc:442] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:18.481824 11462 raft_consensus.cc:482] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:18.482096 11462 raft_consensus.cc:504] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.483863 11462 leader_election.cc:288] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821), 598defb7de294eaa8020b041a87c5a1a (127.8.49.2:41799)
I20260806 16:06:18.489384 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.484638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:57265 (local address 127.8.49.2:41799)
0806 16:06:18.484806 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.484811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.485172 (+ 361us) server_negotiation.cc:407] Connection header received
0806 16:06:18.485434 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.485438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.485507 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.485622 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:18.485998 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.486766 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.488110 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.488578 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.488715 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.488800 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.489054 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.489220 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:06:18.489413 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.484456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:41799 (local address 127.8.49.3:57265)
0806 16:06:18.484986 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:18.485002 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:18.485170 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:18.485667 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:18.485671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:18.485701 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:18.485822 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.485827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.486991 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.486996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.487933 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.487946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.488753 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.488757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.488904 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.488926 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.489023 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.489220 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"spinlock_wait_cycles":27392,"thread_start_us":184,"threads_started":1}
I20260806 16:06:18.490306 11287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.484227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:38821 (local address 127.8.49.3:37073)
0806 16:06:18.484395 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:18.484417 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:18.484565 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:18.485008 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:18.485013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:18.485044 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:18.485237 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.485248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.486582 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.486586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.487838 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.487851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.489725 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.489729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.489865 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.489895 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.490001 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.490134 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 16:06:18.490494 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.484435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:37073 (local address 127.8.49.1:38821)
0806 16:06:18.484649 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.484658 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.484681 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:06:18.484770 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.484773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.484840 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.484946 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:18.485495 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.486416 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.488051 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.489545 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.489680 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.489765 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.490090 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.490331 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 16:06:18.490482 11174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bca24762f3409d965212899e1534a3" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598defb7de294eaa8020b041a87c5a1a" is_pre_election: true
I20260806 16:06:18.490918 11174 raft_consensus.cc:2454] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 0.
I20260806 16:06:18.491182 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bca24762f3409d965212899e1534a3" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047cfb4f2a9e4d088b46199e48bc2474" is_pre_election: true
I20260806 16:06:18.491376 10977 raft_consensus.cc:2454] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 0.
I20260806 16:06:18.491631 11276 leader_election.cc:302] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598defb7de294eaa8020b041a87c5a1a, 98ea9449706a41d89b32b412374c0df8; no voters:
I20260806 16:06:18.492137 11462 raft_consensus.cc:2781] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:18.492251 11462 raft_consensus.cc:482] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:18.492307 11462 raft_consensus.cc:3037] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:18.494184 11462 raft_consensus.cc:504] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.494999 11462 leader_election.cc:288] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 election: Requested vote from peers 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821), 598defb7de294eaa8020b041a87c5a1a (127.8.49.2:41799)
I20260806 16:06:18.497269 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bca24762f3409d965212899e1534a3" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047cfb4f2a9e4d088b46199e48bc2474"
I20260806 16:06:18.497475 10977 raft_consensus.cc:3037] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:18.499301 10977 raft_consensus.cc:2454] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 1.
I20260806 16:06:18.499531 11174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bca24762f3409d965212899e1534a3" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598defb7de294eaa8020b041a87c5a1a"
I20260806 16:06:18.499780 11277 leader_election.cc:302] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047cfb4f2a9e4d088b46199e48bc2474, 98ea9449706a41d89b32b412374c0df8; no voters:
I20260806 16:06:18.499882 11174 raft_consensus.cc:3037] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:18.500077 11462 raft_consensus.cc:2781] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:18.500388 11462 raft_consensus.cc:686] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [term 1 LEADER]: Becoming Leader. State: Replica: 98ea9449706a41d89b32b412374c0df8, State: Running, Role: LEADER
I20260806 16:06:18.500929 11462 consensus_queue.cc:237] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.502377 11174 raft_consensus.cc:2454] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 1.
I20260806 16:06:18.503876 11479 tablet_replica.cc:439] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: TxnStatusTablet state changed. Reason: New leader 98ea9449706a41d89b32b412374c0df8. Latest consensus state: current_term: 1 leader_uuid: "98ea9449706a41d89b32b412374c0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } }
I20260806 16:06:18.504117 11479 tablet_replica.cc:442] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:06:18.504514 11481 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 16:06:18.504849 11481 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 16:06:18.506796 10775 catalog_manager.cc:5495] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98ea9449706a41d89b32b412374c0df8 (127.8.49.3). New cstate: current_term: 1 leader_uuid: "98ea9449706a41d89b32b412374c0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } health_report { overall_health: HEALTHY } } }
I20260806 16:06:18.538151 11474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.532419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:44741 (local address 127.0.0.1:33938)
0806 16:06:18.532708 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:18.532726 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:18.532889 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:18.533439 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:18.533442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:18.533454 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:06:18.533578 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.533586 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.534675 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.534679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.535712 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.535725 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.536482 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.536486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.536591 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.536665 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:06:18.537734 (+ 1069us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:06:18.537754 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.537823 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.537947 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260806 16:06:18.538743 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.532394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.8.49.3:44741)
0806 16:06:18.532544 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.532549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.533033 (+ 484us) server_negotiation.cc:407] Connection header received
0806 16:06:18.533132 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.533136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.533207 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.533299 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:18.533710 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.534517 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.535864 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.536222 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.536328 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.536866 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:18.537559 (+ 693us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:18.537687 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.538415 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.538489 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 16:06:18.548592 10977 raft_consensus.cc:1261] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Refusing update from remote peer 98ea9449706a41d89b32b412374c0df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:18.549130 11174 raft_consensus.cc:1261] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Refusing update from remote peer 98ea9449706a41d89b32b412374c0df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:18.549520 11479 consensus_queue.cc:1035] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:18.549990 11462 consensus_queue.cc:1035] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:18.555979 11460 tablet_replica.cc:439] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474: TxnStatusTablet state changed. Reason: New leader 98ea9449706a41d89b32b412374c0df8. Latest consensus state: current_term: 1 leader_uuid: "98ea9449706a41d89b32b412374c0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } }
I20260806 16:06:18.556169 11460 tablet_replica.cc:442] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:18.559777 11461 tablet_replica.cc:439] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a: TxnStatusTablet state changed. Reason: New leader 98ea9449706a41d89b32b412374c0df8. Latest consensus state: current_term: 1 leader_uuid: "98ea9449706a41d89b32b412374c0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } }
I20260806 16:06:18.560073 11461 tablet_replica.cc:442] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:18.565508 11460 tablet_replica.cc:439] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98ea9449706a41d89b32b412374c0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } }
I20260806 16:06:18.565768 11460 tablet_replica.cc:442] T 14bca24762f3409d965212899e1534a3 P 047cfb4f2a9e4d088b46199e48bc2474: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:18.576428 11461 tablet_replica.cc:439] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98ea9449706a41d89b32b412374c0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } }
I20260806 16:06:18.576776 11461 tablet_replica.cc:442] T 14bca24762f3409d965212899e1534a3 P 598defb7de294eaa8020b041a87c5a1a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:18.576974 11479 tablet_replica.cc:439] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98ea9449706a41d89b32b412374c0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } }
I20260806 16:06:18.577147 11479 tablet_replica.cc:442] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:06:18.581144 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.575130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58626 (local address 127.8.49.2:41799)
0806 16:06:18.575334 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.575339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.575942 (+ 603us) server_negotiation.cc:407] Connection header received
0806 16:06:18.576215 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.576220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.576293 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.576391 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:18.576878 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.577674 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.579023 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.579392 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.579474 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.579909 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:18.580594 (+ 685us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:18.580693 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.580894 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.580976 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 16:06:18.583926 11462 tablet_replica.cc:439] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98ea9449706a41d89b32b412374c0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } } }
I20260806 16:06:18.584303 11462 tablet_replica.cc:442] T 14bca24762f3409d965212899e1534a3 P 98ea9449706a41d89b32b412374c0df8: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:06:18.585191 11484 mvcc.cc:205] Tried to move back new op lower bound from 7315588622525054976 to 7315588622344417280. Current Snapshot: MvccSnapshot[applied={T|T < 7315588622525054976 or (T in {7315588622525054976})}]
I20260806 16:06:18.598407 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.592810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:44741 (local address 127.8.49.2:42343)
0806 16:06:18.593317 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:18.593331 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:18.593482 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:18.593773 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:18.593776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:18.593784 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:06:18.593940 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.593945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.595550 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.595557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.596503 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.596520 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.597139 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.597143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.597259 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.597331 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:06:18.597767 (+ 436us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:06:18.597776 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.597874 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.598189 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":98,"threads_started":1}
I20260806 16:06:18.598352 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.592922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:42343 (local address 127.8.49.3:44741)
0806 16:06:18.593085 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.593090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.593425 (+ 335us) server_negotiation.cc:407] Connection header received
0806 16:06:18.593590 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.593593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.593666 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.593757 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:18.594165 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.595325 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.596676 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.596985 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.597126 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.597544 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:18.597654 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:18.597748 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.597891 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.598155 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:06:18.608871 10977 raft_consensus.cc:1261] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Refusing update from remote peer 598defb7de294eaa8020b041a87c5a1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:18.609207 11351 raft_consensus.cc:1261] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Refusing update from remote peer 598defb7de294eaa8020b041a87c5a1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:18.609599 11465 consensus_queue.cc:1035] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:18.609813 11461 consensus_queue.cc:1035] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:18.614921 11489 mvcc.cc:205] Tried to move back new op lower bound from 7315588622777126912 to 7315588621765734400. Current Snapshot: MvccSnapshot[applied={T|T < 7315588622777126912}]
I20260806 16:06:18.622835 11485 mvcc.cc:205] Tried to move back new op lower bound from 7315588622777126912 to 7315588621765734400. Current Snapshot: MvccSnapshot[applied={T|T < 7315588622777126912}]
I20260806 16:06:18.647662 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.639912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:43931 (local address 127.8.49.2:41799)
0806 16:06:18.640260 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:18.640305 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:18.643348 (+ 3043us) server_negotiation.cc:407] Connection header received
0806 16:06:18.643642 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:18.643645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:18.643718 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:18.643776 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:18.644403 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.645185 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.646120 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.646397 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.646526 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.647100 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:18.647207 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:18.647278 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:18.647439 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 16:06:18.647510 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215}
I20260806 16:06:18.648708 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:18.639756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:41799 (local address 127.8.49.3:43931)
0806 16:06:18.643231 (+ 3475us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:18.643254 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:18.643482 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:18.644060 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:18.644065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:18.644074 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:06:18.644204 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.644213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.645324 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.645326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.645983 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:18.645990 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:18.646562 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:18.646567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:18.646650 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:18.646712 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:06:18.647323 (+ 611us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:06:18.647331 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 16:06:18.648400 (+ 1069us) client_negotiation.cc:240] Negotiation successful
0806 16:06:18.648512 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3276,"thread_start_us":125,"threads_started":1}
I20260806 16:06:18.712333 10773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:18.712590 10773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:18.717468 10773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f: Sending DeleteTablet for 3 replicas of tablet b882c5a8f8ac4b1387c753544d08a276
I20260806 16:06:18.721679 11336 tablet_service.cc:1514] Processing DeleteTablet for tablet b882c5a8f8ac4b1387c753544d08a276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:18 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:33928
I20260806 16:06:18.722787 11149 tablet_service.cc:1514] Processing DeleteTablet for tablet b882c5a8f8ac4b1387c753544d08a276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:18 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:45738
I20260806 16:06:18.726055 10775 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:18.726928 10775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:18.727659 11519 tablet_replica.cc:330] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: stopping tablet replica
I20260806 16:06:18.728276 11520 tablet_replica.cc:330] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: stopping tablet replica
I20260806 16:06:18.728262 11519 raft_consensus.cc:2229] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:18.728593 11520 raft_consensus.cc:2229] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:18.728796 11519 raft_consensus.cc:2258] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:18.730403 10954 tablet_service.cc:1514] Processing DeleteTablet for tablet b882c5a8f8ac4b1387c753544d08a276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:18 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:50826
I20260806 16:06:18.731261 11519 ts_tablet_manager.cc:1905] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:18.729058 11520 raft_consensus.cc:2258] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:18.736816 11519 ts_tablet_manager.cc:1918] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:18.737179 11519 log.cc:1196] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/b882c5a8f8ac4b1387c753544d08a276
I20260806 16:06:18.737643 11519 ts_tablet_manager.cc:1939] T b882c5a8f8ac4b1387c753544d08a276 P 98ea9449706a41d89b32b412374c0df8: Deleting consensus metadata
I20260806 16:06:18.738229 11521 tablet_replica.cc:330] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: stopping tablet replica
I20260806 16:06:18.739015 11520 ts_tablet_manager.cc:1905] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:18.739514 11521 raft_consensus.cc:2229] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:18.740908 11521 raft_consensus.cc:2258] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:18.743233 11336 tablet_service.cc:1467] Processing CreateTablet for tablet 2be605d96c1c4f81bfc8a95ae25c346a (DEFAULT_TABLE table=test-table [id=46be6ae48ee4442aa4c0b79ff9319453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:18.743815 11336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be605d96c1c4f81bfc8a95ae25c346a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:18.745270 11520 ts_tablet_manager.cc:1918] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:18.745599 11520 log.cc:1196] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/b882c5a8f8ac4b1387c753544d08a276
I20260806 16:06:18.746071 11520 ts_tablet_manager.cc:1939] T b882c5a8f8ac4b1387c753544d08a276 P 598defb7de294eaa8020b041a87c5a1a: Deleting consensus metadata
I20260806 16:06:18.747586 10954 tablet_service.cc:1467] Processing CreateTablet for tablet 2be605d96c1c4f81bfc8a95ae25c346a (DEFAULT_TABLE table=test-table [id=46be6ae48ee4442aa4c0b79ff9319453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:18.747830 10755 catalog_manager.cc:4841] TS 598defb7de294eaa8020b041a87c5a1a (127.8.49.2:41799): tablet b882c5a8f8ac4b1387c753544d08a276 (table test-table [id=e809b2bf57f54dd5aa93baff7fff8b60]) successfully deleted
I20260806 16:06:18.748188 10954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be605d96c1c4f81bfc8a95ae25c346a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:18.748688 11521 ts_tablet_manager.cc:1905] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:18.750982 11149 tablet_service.cc:1467] Processing CreateTablet for tablet 2be605d96c1c4f81bfc8a95ae25c346a (DEFAULT_TABLE table=test-table [id=46be6ae48ee4442aa4c0b79ff9319453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:18.751662 11149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2be605d96c1c4f81bfc8a95ae25c346a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:18.753477 11521 ts_tablet_manager.cc:1918] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:18.753660 11521 log.cc:1196] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/wal/wals/b882c5a8f8ac4b1387c753544d08a276
I20260806 16:06:18.754070 11521 ts_tablet_manager.cc:1939] T b882c5a8f8ac4b1387c753544d08a276 P 047cfb4f2a9e4d088b46199e48bc2474: Deleting consensus metadata
I20260806 16:06:18.756414 10758 catalog_manager.cc:4841] TS 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821): tablet b882c5a8f8ac4b1387c753544d08a276 (table test-table [id=e809b2bf57f54dd5aa93baff7fff8b60]) successfully deleted
I20260806 16:06:18.763356 11453 tablet_bootstrap.cc:492] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a: Bootstrap starting.
I20260806 16:06:18.764801 11453 tablet_bootstrap.cc:654] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:18.765360 10758 catalog_manager.cc:4841] TS 98ea9449706a41d89b32b412374c0df8 (127.8.49.3:44741): tablet b882c5a8f8ac4b1387c753544d08a276 (table test-table [id=e809b2bf57f54dd5aa93baff7fff8b60]) successfully deleted
I20260806 16:06:18.766767 11453 tablet_bootstrap.cc:492] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a: No bootstrap required, opened a new log
I20260806 16:06:18.766922 11453 ts_tablet_manager.cc:1397] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260806 16:06:18.767475 11453 raft_consensus.cc:348] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.767637 11453 raft_consensus.cc:374] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:18.767680 11453 raft_consensus.cc:729] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598defb7de294eaa8020b041a87c5a1a, State: Initialized, Role: FOLLOWER
I20260806 16:06:18.767830 11453 consensus_queue.cc:260] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.768131 11453 ts_tablet_manager.cc:1428] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 16:06:18.771337 11450 tablet_bootstrap.cc:492] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474: Bootstrap starting.
I20260806 16:06:18.773130 11450 tablet_bootstrap.cc:654] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:18.775522 11455 tablet_bootstrap.cc:492] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8: Bootstrap starting.
I20260806 16:06:18.777447 11455 tablet_bootstrap.cc:654] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:18.780627 11455 tablet_bootstrap.cc:492] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8: No bootstrap required, opened a new log
I20260806 16:06:18.780654 11450 tablet_bootstrap.cc:492] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474: No bootstrap required, opened a new log
I20260806 16:06:18.780768 11455 ts_tablet_manager.cc:1397] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 16:06:18.780830 11450 ts_tablet_manager.cc:1397] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260806 16:06:18.781381 11455 raft_consensus.cc:348] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.781704 11455 raft_consensus.cc:374] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:18.781757 11455 raft_consensus.cc:729] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ea9449706a41d89b32b412374c0df8, State: Initialized, Role: FOLLOWER
I20260806 16:06:18.781868 11450 raft_consensus.cc:348] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.782034 11450 raft_consensus.cc:374] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:18.781992 11455 consensus_queue.cc:260] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.782073 11450 raft_consensus.cc:729] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047cfb4f2a9e4d088b46199e48bc2474, State: Initialized, Role: FOLLOWER
I20260806 16:06:18.782320 11455 ts_tablet_manager.cc:1428] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:06:18.782464 11450 consensus_queue.cc:260] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.782927 11450 ts_tablet_manager.cc:1428] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 16:06:18.872303 11465 raft_consensus.cc:482] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:18.872560 11465 raft_consensus.cc:504] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.873370 11465 leader_election.cc:288] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821), 98ea9449706a41d89b32b412374c0df8 (127.8.49.3:44741)
I20260806 16:06:18.874145 11351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be605d96c1c4f81bfc8a95ae25c346a" candidate_uuid: "598defb7de294eaa8020b041a87c5a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ea9449706a41d89b32b412374c0df8" is_pre_election: true
I20260806 16:06:18.874190 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be605d96c1c4f81bfc8a95ae25c346a" candidate_uuid: "598defb7de294eaa8020b041a87c5a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047cfb4f2a9e4d088b46199e48bc2474" is_pre_election: true
I20260806 16:06:18.874433 10977 raft_consensus.cc:2454] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 598defb7de294eaa8020b041a87c5a1a in term 0.
I20260806 16:06:18.874387 11351 raft_consensus.cc:2454] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 598defb7de294eaa8020b041a87c5a1a in term 0.
I20260806 16:06:18.875128 11086 leader_election.cc:302] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598defb7de294eaa8020b041a87c5a1a, 98ea9449706a41d89b32b412374c0df8; no voters:
I20260806 16:06:18.875633 11465 raft_consensus.cc:2781] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:18.875721 11465 raft_consensus.cc:482] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:18.875772 11465 raft_consensus.cc:3037] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:18.877895 11465 raft_consensus.cc:504] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.878600 11465 leader_election.cc:288] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [CANDIDATE]: Term 1 election: Requested vote from peers 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821), 98ea9449706a41d89b32b412374c0df8 (127.8.49.3:44741)
I20260806 16:06:18.879209 11351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be605d96c1c4f81bfc8a95ae25c346a" candidate_uuid: "598defb7de294eaa8020b041a87c5a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98ea9449706a41d89b32b412374c0df8"
I20260806 16:06:18.879220 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2be605d96c1c4f81bfc8a95ae25c346a" candidate_uuid: "598defb7de294eaa8020b041a87c5a1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047cfb4f2a9e4d088b46199e48bc2474"
I20260806 16:06:18.879441 10977 raft_consensus.cc:3037] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:18.879508 11351 raft_consensus.cc:3037] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:18.881860 11351 raft_consensus.cc:2454] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 598defb7de294eaa8020b041a87c5a1a in term 1.
I20260806 16:06:18.882589 10977 raft_consensus.cc:2454] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 598defb7de294eaa8020b041a87c5a1a in term 1.
I20260806 16:06:18.882632 11086 leader_election.cc:302] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598defb7de294eaa8020b041a87c5a1a, 98ea9449706a41d89b32b412374c0df8; no voters:
I20260806 16:06:18.882925 11461 raft_consensus.cc:2781] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:18.883025 11461 raft_consensus.cc:686] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 1 LEADER]: Becoming Leader. State: Replica: 598defb7de294eaa8020b041a87c5a1a, State: Running, Role: LEADER
I20260806 16:06:18.883212 11461 consensus_queue.cc:237] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:18.886538 10773 catalog_manager.cc:5495] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 598defb7de294eaa8020b041a87c5a1a (127.8.49.2). New cstate: current_term: 1 leader_uuid: "598defb7de294eaa8020b041a87c5a1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } health_report { overall_health: UNKNOWN } } }
I20260806 16:06:18.929688 10977 raft_consensus.cc:1261] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Refusing update from remote peer 598defb7de294eaa8020b041a87c5a1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:18.929754 11351 raft_consensus.cc:1261] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Refusing update from remote peer 598defb7de294eaa8020b041a87c5a1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:18.930612 11465 consensus_queue.cc:1035] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:18.930857 11461 consensus_queue.cc:1035] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 16:06:18.953018 11489 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 16:06:20.558480 10775 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:20.558738 10775 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:20.563242 10775 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f: Sending DeleteTablet for 3 replicas of tablet 2be605d96c1c4f81bfc8a95ae25c346a
I20260806 16:06:20.564430 11336 tablet_service.cc:1514] Processing DeleteTablet for tablet 2be605d96c1c4f81bfc8a95ae25c346a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:20 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:33928
I20260806 16:06:20.564419 10954 tablet_service.cc:1514] Processing DeleteTablet for tablet 2be605d96c1c4f81bfc8a95ae25c346a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:20 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:50826
I20260806 16:06:20.565694 11563 tablet_replica.cc:330] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8: stopping tablet replica
I20260806 16:06:20.565907 11563 raft_consensus.cc:2229] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:20.566304 11564 tablet_replica.cc:330] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474: stopping tablet replica
I20260806 16:06:20.566476 11564 raft_consensus.cc:2229] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:20.566638 11564 raft_consensus.cc:2258] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:20.566618 11149 tablet_service.cc:1514] Processing DeleteTablet for tablet 2be605d96c1c4f81bfc8a95ae25c346a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:20 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:45738
I20260806 16:06:20.566869 11563 raft_consensus.cc:2258] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:20.567655 11564 ts_tablet_manager.cc:1905] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:20.569815 11567 tablet_replica.cc:330] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a: stopping tablet replica
I20260806 16:06:20.569923 11563 ts_tablet_manager.cc:1905] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:20.570011 11567 raft_consensus.cc:2229] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:20.570307 11567 raft_consensus.cc:2258] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:20.571131 11567 ts_tablet_manager.cc:1905] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 16:06:20 dist-test-slave-7z32 krb5kdc[10712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032380, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260806 16:06:20.657927 11564 ts_tablet_manager.cc:1918] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 16:06:20.658142 11564 log.cc:1196] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/wal/wals/2be605d96c1c4f81bfc8a95ae25c346a
I20260806 16:06:20.658166 11563 ts_tablet_manager.cc:1918] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 16:06:20.658319 11563 log.cc:1196] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/2be605d96c1c4f81bfc8a95ae25c346a
I20260806 16:06:20.658679 11564 ts_tablet_manager.cc:1939] T 2be605d96c1c4f81bfc8a95ae25c346a P 047cfb4f2a9e4d088b46199e48bc2474: Deleting consensus metadata
I20260806 16:06:20.658794 11563 ts_tablet_manager.cc:1939] T 2be605d96c1c4f81bfc8a95ae25c346a P 98ea9449706a41d89b32b412374c0df8: Deleting consensus metadata
Aug 06 16:06:20 dist-test-slave-7z32 krb5kdc[10712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032380, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:20.673754 11578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:20.659547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38870 (local address 127.8.49.62:33693)
0806 16:06:20.660045 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:20.660057 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:20.660603 (+ 546us) server_negotiation.cc:407] Connection header received
0806 16:06:20.660802 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:20.660806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:20.660923 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:20.661054 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:20.662326 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:20.663385 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:20.664454 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:20.664719 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:20.664890 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:20.668405 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:20.668409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:20.668411 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:20.668457 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:20.670971 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:20.671616 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:20.671623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:20.671629 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:20.671859 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:20.672382 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:20.672389 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:20.672390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:20.672893 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:20.673044 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:20.673385 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 16:06:20.673547 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":158,"threads_started":1}
I20260806 16:06:20.678100 10775 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:20.678803 10775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:20.686483 10758 catalog_manager.cc:4841] TS 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821): tablet 2be605d96c1c4f81bfc8a95ae25c346a (table test-table [id=46be6ae48ee4442aa4c0b79ff9319453]) successfully deleted
I20260806 16:06:20.686514 11336 tablet_service.cc:1467] Processing CreateTablet for tablet 8b087e56ec5b45f89a8e32b5aa710dbf (DEFAULT_TABLE table=test-table [id=706cbbb1827442ff9fdf1311306b1837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:20.686514 11149 tablet_service.cc:1467] Processing CreateTablet for tablet 8b087e56ec5b45f89a8e32b5aa710dbf (DEFAULT_TABLE table=test-table [id=706cbbb1827442ff9fdf1311306b1837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:20.686705 10954 tablet_service.cc:1467] Processing CreateTablet for tablet 8b087e56ec5b45f89a8e32b5aa710dbf (DEFAULT_TABLE table=test-table [id=706cbbb1827442ff9fdf1311306b1837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:20.686976 11336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b087e56ec5b45f89a8e32b5aa710dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:20.687196 10954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b087e56ec5b45f89a8e32b5aa710dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:20.688503 10758 catalog_manager.cc:4841] TS 98ea9449706a41d89b32b412374c0df8 (127.8.49.3:44741): tablet 2be605d96c1c4f81bfc8a95ae25c346a (table test-table [id=46be6ae48ee4442aa4c0b79ff9319453]) successfully deleted
I20260806 16:06:20.689867 11580 tablet_bootstrap.cc:492] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8: Bootstrap starting.
I20260806 16:06:20.691109 11580 tablet_bootstrap.cc:654] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:20.691282 11581 tablet_bootstrap.cc:492] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474: Bootstrap starting.
I20260806 16:06:20.693439 11581 tablet_bootstrap.cc:654] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:20.693389 11580 tablet_bootstrap.cc:492] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8: No bootstrap required, opened a new log
I20260806 16:06:20.693608 11580 ts_tablet_manager.cc:1397] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:20.694293 11580 raft_consensus.cc:348] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:20.694468 11580 raft_consensus.cc:374] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:20.694514 11580 raft_consensus.cc:729] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ea9449706a41d89b32b412374c0df8, State: Initialized, Role: FOLLOWER
I20260806 16:06:20.694656 11580 consensus_queue.cc:260] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:20.695014 11580 ts_tablet_manager.cc:1428] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:06:20.696144 11581 tablet_bootstrap.cc:492] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474: No bootstrap required, opened a new log
I20260806 16:06:20.696290 11581 ts_tablet_manager.cc:1397] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 16:06:20.696830 11581 raft_consensus.cc:348] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:20.696962 11581 raft_consensus.cc:374] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:20.696995 11581 raft_consensus.cc:729] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047cfb4f2a9e4d088b46199e48bc2474, State: Initialized, Role: FOLLOWER
I20260806 16:06:20.697142 11581 consensus_queue.cc:260] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:20.697589 11581 ts_tablet_manager.cc:1428] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:06:20.698125 11567 ts_tablet_manager.cc:1918] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 16:06:20.698269 11567 log.cc:1196] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/2be605d96c1c4f81bfc8a95ae25c346a
I20260806 16:06:20.698750 11567 ts_tablet_manager.cc:1939] T 2be605d96c1c4f81bfc8a95ae25c346a P 598defb7de294eaa8020b041a87c5a1a: Deleting consensus metadata
I20260806 16:06:20.699136 11149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b087e56ec5b45f89a8e32b5aa710dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:20.702886 11585 tablet_bootstrap.cc:492] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a: Bootstrap starting.
I20260806 16:06:20.704274 11585 tablet_bootstrap.cc:654] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:20.707422 11585 tablet_bootstrap.cc:492] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a: No bootstrap required, opened a new log
I20260806 16:06:20.707584 11585 ts_tablet_manager.cc:1397] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 16:06:20.708421 11585 raft_consensus.cc:348] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:20.708618 11585 raft_consensus.cc:374] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:20.708663 11585 raft_consensus.cc:729] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598defb7de294eaa8020b041a87c5a1a, State: Initialized, Role: FOLLOWER
I20260806 16:06:20.708858 11585 consensus_queue.cc:260] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:20.709357 11585 ts_tablet_manager.cc:1428] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 16:06:20.729862 10755 catalog_manager.cc:4841] TS 598defb7de294eaa8020b041a87c5a1a (127.8.49.2:41799): tablet 2be605d96c1c4f81bfc8a95ae25c346a (table test-table [id=46be6ae48ee4442aa4c0b79ff9319453]) successfully deleted
I20260806 16:06:20.811298 11546 raft_consensus.cc:482] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:20.811554 11546 raft_consensus.cc:504] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:20.812173 11546 leader_election.cc:288] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821), 598defb7de294eaa8020b041a87c5a1a (127.8.49.2:41799)
I20260806 16:06:20.813145 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b087e56ec5b45f89a8e32b5aa710dbf" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047cfb4f2a9e4d088b46199e48bc2474" is_pre_election: true
I20260806 16:06:20.813292 11173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b087e56ec5b45f89a8e32b5aa710dbf" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598defb7de294eaa8020b041a87c5a1a" is_pre_election: true
I20260806 16:06:20.813446 10977 raft_consensus.cc:2454] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 0.
I20260806 16:06:20.813513 11173 raft_consensus.cc:2454] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 0.
I20260806 16:06:20.814013 11276 leader_election.cc:302] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598defb7de294eaa8020b041a87c5a1a, 98ea9449706a41d89b32b412374c0df8; no voters:
I20260806 16:06:20.814284 11546 raft_consensus.cc:2781] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:20.814361 11546 raft_consensus.cc:482] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:20.814393 11546 raft_consensus.cc:3037] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:20.858338 11546 raft_consensus.cc:504] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:20.859277 11546 leader_election.cc:288] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 election: Requested vote from peers 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821), 598defb7de294eaa8020b041a87c5a1a (127.8.49.2:41799)
I20260806 16:06:20.860004 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b087e56ec5b45f89a8e32b5aa710dbf" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047cfb4f2a9e4d088b46199e48bc2474"
I20260806 16:06:20.860066 11173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b087e56ec5b45f89a8e32b5aa710dbf" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598defb7de294eaa8020b041a87c5a1a"
I20260806 16:06:20.860255 10977 raft_consensus.cc:3037] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:20.860280 11173 raft_consensus.cc:3037] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:20.862131 10977 raft_consensus.cc:2454] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 1.
I20260806 16:06:20.862690 11277 leader_election.cc:302] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047cfb4f2a9e4d088b46199e48bc2474, 98ea9449706a41d89b32b412374c0df8; no voters:
I20260806 16:06:20.862814 11173 raft_consensus.cc:2454] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 1.
I20260806 16:06:20.863034 11546 raft_consensus.cc:2781] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:20.863165 11546 raft_consensus.cc:686] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 1 LEADER]: Becoming Leader. State: Replica: 98ea9449706a41d89b32b412374c0df8, State: Running, Role: LEADER
I20260806 16:06:20.863377 11546 consensus_queue.cc:237] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:20.866469 10773 catalog_manager.cc:5495] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98ea9449706a41d89b32b412374c0df8 (127.8.49.3). New cstate: current_term: 1 leader_uuid: "98ea9449706a41d89b32b412374c0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } health_report { overall_health: HEALTHY } } }
I20260806 16:06:20.915303 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:20.910468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45766 (local address 127.8.49.3:44741)
0806 16:06:20.910967 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:20.910977 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:20.911013 (+ 36us) server_negotiation.cc:407] Connection header received
0806 16:06:20.911081 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:20.911084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:20.911147 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:20.911265 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:20.911780 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:20.912489 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:20.913651 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:20.913938 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:20.914052 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:20.914464 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:20.914594 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:20.914757 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:20.914932 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 16:06:20.915023 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":196,"threads_started":1}
I20260806 16:06:20.931115 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:20.926034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:32803 (local address 127.8.49.3:44741)
0806 16:06:20.926245 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:20.926256 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:20.926445 (+ 189us) server_negotiation.cc:407] Connection header received
0806 16:06:20.926615 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:20.926619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:20.926681 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:20.926895 (+ 214us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:20.927340 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:20.928022 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:20.929128 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:20.929611 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:20.930115 (+ 504us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:20.930456 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:20.930565 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:20.930684 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:20.930747 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 16:06:20.930932 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 16:06:20.931135 11592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:20.925917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:44741 (local address 127.8.49.3:32803)
0806 16:06:20.926298 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:20.926312 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:20.926439 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:20.926991 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:20.926994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:20.927004 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:06:20.927135 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:20.927141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:20.928217 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:20.928221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:20.928937 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:20.928949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:20.930174 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:20.930177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:20.930255 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:20.930294 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:06:20.930660 (+ 366us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:06:20.930666 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 16:06:20.930733 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 16:06:20.930896 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":161,"threads_started":1}
I20260806 16:06:20.939687 11173 raft_consensus.cc:1261] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Refusing update from remote peer 98ea9449706a41d89b32b412374c0df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:20.939924 10977 raft_consensus.cc:1261] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Refusing update from remote peer 98ea9449706a41d89b32b412374c0df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:20.940321 11479 consensus_queue.cc:1035] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:20.940642 11546 consensus_queue.cc:1035] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:20.996335 10775 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:20.996573 10775 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:21.000168 10775 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f: Sending DeleteTablet for 3 replicas of tablet 8b087e56ec5b45f89a8e32b5aa710dbf
I20260806 16:06:21.001852 10954 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b087e56ec5b45f89a8e32b5aa710dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:20 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:50826
I20260806 16:06:21.002171 11564 tablet_replica.cc:330] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474: stopping tablet replica
I20260806 16:06:21.002494 11564 raft_consensus.cc:2229] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:21.002739 11564 raft_consensus.cc:2258] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:21.003607 11149 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b087e56ec5b45f89a8e32b5aa710dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:20 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:45738
I20260806 16:06:21.003983 11567 tablet_replica.cc:330] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a: stopping tablet replica
I20260806 16:06:21.004169 11567 raft_consensus.cc:2229] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:21.004350 11567 raft_consensus.cc:2258] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:21.004706 10775 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 16:06:21.005409 11564 ts_tablet_manager.cc:1905] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 16:06:21.005441 10775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:21.005580 11567 ts_tablet_manager.cc:1905] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:21.007228 11336 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b087e56ec5b45f89a8e32b5aa710dbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:20 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:33928
I20260806 16:06:21.007491 11563 tablet_replica.cc:330] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8: stopping tablet replica
I20260806 16:06:21.007651 11563 raft_consensus.cc:2229] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:21.008168 11563 raft_consensus.cc:2258] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:21.009836 11563 ts_tablet_manager.cc:1905] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:21.017378 10954 tablet_service.cc:1467] Processing CreateTablet for tablet 3e45c63a79464000a72d83c1cd231234 (DEFAULT_TABLE table=test-table [id=ec32b2d1b6d049ab8b7884ae5bf7b2a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:21.018555 11149 tablet_service.cc:1467] Processing CreateTablet for tablet 3e45c63a79464000a72d83c1cd231234 (DEFAULT_TABLE table=test-table [id=ec32b2d1b6d049ab8b7884ae5bf7b2a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:21.020846 11336 tablet_service.cc:1467] Processing CreateTablet for tablet 3e45c63a79464000a72d83c1cd231234 (DEFAULT_TABLE table=test-table [id=ec32b2d1b6d049ab8b7884ae5bf7b2a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:21.065935 10954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e45c63a79464000a72d83c1cd231234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:21.068516 11564 ts_tablet_manager.cc:1918] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:21.068660 11564 log.cc:1196] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-0/wal/wals/8b087e56ec5b45f89a8e32b5aa710dbf
I20260806 16:06:21.069016 11564 ts_tablet_manager.cc:1939] T 8b087e56ec5b45f89a8e32b5aa710dbf P 047cfb4f2a9e4d088b46199e48bc2474: Deleting consensus metadata
I20260806 16:06:21.069764 11149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e45c63a79464000a72d83c1cd231234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:21.072466 11567 ts_tablet_manager.cc:1918] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:21.072587 11567 log.cc:1196] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/8b087e56ec5b45f89a8e32b5aa710dbf
I20260806 16:06:21.072916 11567 ts_tablet_manager.cc:1939] T 8b087e56ec5b45f89a8e32b5aa710dbf P 598defb7de294eaa8020b041a87c5a1a: Deleting consensus metadata
I20260806 16:06:21.073935 11581 tablet_bootstrap.cc:492] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474: Bootstrap starting.
I20260806 16:06:21.075120 11336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e45c63a79464000a72d83c1cd231234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:21.075392 11581 tablet_bootstrap.cc:654] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:21.076664 11585 tablet_bootstrap.cc:492] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a: Bootstrap starting.
I20260806 16:06:21.078161 11581 tablet_bootstrap.cc:492] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474: No bootstrap required, opened a new log
I20260806 16:06:21.078274 11585 tablet_bootstrap.cc:654] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:21.078286 11581 ts_tablet_manager.cc:1397] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:21.079018 11581 raft_consensus.cc:348] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:21.079178 11581 raft_consensus.cc:374] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:21.079212 11581 raft_consensus.cc:729] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047cfb4f2a9e4d088b46199e48bc2474, State: Initialized, Role: FOLLOWER
I20260806 16:06:21.079341 11581 consensus_queue.cc:260] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:21.079641 11581 ts_tablet_manager.cc:1428] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:06:21.080083 11580 tablet_bootstrap.cc:492] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8: Bootstrap starting.
I20260806 16:06:21.081239 11580 tablet_bootstrap.cc:654] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:21.083797 11585 tablet_bootstrap.cc:492] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a: No bootstrap required, opened a new log
I20260806 16:06:21.083973 11585 ts_tablet_manager.cc:1397] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260806 16:06:21.084561 11580 tablet_bootstrap.cc:492] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8: No bootstrap required, opened a new log
I20260806 16:06:21.084569 11585 raft_consensus.cc:348] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:21.084699 11580 ts_tablet_manager.cc:1397] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 16:06:21.084739 11585 raft_consensus.cc:374] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:21.084777 11585 raft_consensus.cc:729] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598defb7de294eaa8020b041a87c5a1a, State: Initialized, Role: FOLLOWER
I20260806 16:06:21.084978 11585 consensus_queue.cc:260] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:21.085282 11580 raft_consensus.cc:348] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:21.085415 11580 raft_consensus.cc:374] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:21.085446 11580 raft_consensus.cc:729] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ea9449706a41d89b32b412374c0df8, State: Initialized, Role: FOLLOWER
I20260806 16:06:21.085656 11580 consensus_queue.cc:260] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:21.086138 11585 ts_tablet_manager.cc:1428] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:21.086338 11580 ts_tablet_manager.cc:1428] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:21.089612 10758 catalog_manager.cc:4841] TS 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821): tablet 8b087e56ec5b45f89a8e32b5aa710dbf (table test-table [id=706cbbb1827442ff9fdf1311306b1837]) successfully deleted
I20260806 16:06:21.107806 11479 raft_consensus.cc:482] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:21.108044 11479 raft_consensus.cc:504] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:21.108743 11479 leader_election.cc:288] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821), 598defb7de294eaa8020b041a87c5a1a (127.8.49.2:41799)
I20260806 16:06:21.109406 11174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e45c63a79464000a72d83c1cd231234" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598defb7de294eaa8020b041a87c5a1a" is_pre_election: true
I20260806 16:06:21.109609 11174 raft_consensus.cc:2454] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 0.
I20260806 16:06:21.109506 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e45c63a79464000a72d83c1cd231234" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047cfb4f2a9e4d088b46199e48bc2474" is_pre_election: true
I20260806 16:06:21.109772 10977 raft_consensus.cc:2454] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 0.
I20260806 16:06:21.110066 11276 leader_election.cc:302] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598defb7de294eaa8020b041a87c5a1a, 98ea9449706a41d89b32b412374c0df8; no voters:
I20260806 16:06:21.110278 11479 raft_consensus.cc:2781] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:21.110368 11479 raft_consensus.cc:482] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:21.110424 11479 raft_consensus.cc:3037] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:21.130092 10755 catalog_manager.cc:4841] TS 598defb7de294eaa8020b041a87c5a1a (127.8.49.2:41799): tablet 8b087e56ec5b45f89a8e32b5aa710dbf (table test-table [id=706cbbb1827442ff9fdf1311306b1837]) successfully deleted
I20260806 16:06:21.136230 11563 ts_tablet_manager.cc:1918] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:21.136416 11563 log.cc:1196] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/8b087e56ec5b45f89a8e32b5aa710dbf
I20260806 16:06:21.136821 11563 ts_tablet_manager.cc:1939] T 8b087e56ec5b45f89a8e32b5aa710dbf P 98ea9449706a41d89b32b412374c0df8: Deleting consensus metadata
I20260806 16:06:21.175181 11479 raft_consensus.cc:504] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:21.176478 10758 catalog_manager.cc:4841] TS 98ea9449706a41d89b32b412374c0df8 (127.8.49.3:44741): tablet 8b087e56ec5b45f89a8e32b5aa710dbf (table test-table [id=706cbbb1827442ff9fdf1311306b1837]) successfully deleted
I20260806 16:06:21.176916 11174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e45c63a79464000a72d83c1cd231234" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598defb7de294eaa8020b041a87c5a1a"
I20260806 16:06:21.177111 11174 raft_consensus.cc:3037] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:21.177045 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e45c63a79464000a72d83c1cd231234" candidate_uuid: "98ea9449706a41d89b32b412374c0df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047cfb4f2a9e4d088b46199e48bc2474"
I20260806 16:06:21.177338 10977 raft_consensus.cc:3037] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:21.178959 11174 raft_consensus.cc:2454] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 1.
I20260806 16:06:21.179036 10977 raft_consensus.cc:2454] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ea9449706a41d89b32b412374c0df8 in term 1.
I20260806 16:06:21.179549 11277 leader_election.cc:302] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047cfb4f2a9e4d088b46199e48bc2474, 98ea9449706a41d89b32b412374c0df8; no voters:
I20260806 16:06:21.179885 11479 leader_election.cc:288] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [CANDIDATE]: Term 1 election: Requested vote from peers 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821), 598defb7de294eaa8020b041a87c5a1a (127.8.49.2:41799)
I20260806 16:06:21.180145 11546 raft_consensus.cc:2781] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:21.180295 11546 raft_consensus.cc:686] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 1 LEADER]: Becoming Leader. State: Replica: 98ea9449706a41d89b32b412374c0df8, State: Running, Role: LEADER
I20260806 16:06:21.180481 11546 consensus_queue.cc:237] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } }
I20260806 16:06:21.183857 10773 catalog_manager.cc:5495] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98ea9449706a41d89b32b412374c0df8 (127.8.49.3). New cstate: current_term: 1 leader_uuid: "98ea9449706a41d89b32b412374c0df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98ea9449706a41d89b32b412374c0df8" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44741 } health_report { overall_health: HEALTHY } } }
I20260806 16:06:21.216488 11174 raft_consensus.cc:1261] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Refusing update from remote peer 98ea9449706a41d89b32b412374c0df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:21.216508 10977 raft_consensus.cc:1261] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Refusing update from remote peer 98ea9449706a41d89b32b412374c0df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:21.217008 11546 consensus_queue.cc:1035] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "047cfb4f2a9e4d088b46199e48bc2474" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:21.217240 11479 consensus_queue.cc:1035] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "598defb7de294eaa8020b041a87c5a1a" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 16:06:21.239058 11537 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 16:06:22.839330 10773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:22.839599 10773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:22.844645 10773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dba7ac7e6f94be6810be0e354bc972f: Sending DeleteTablet for 3 replicas of tablet 3e45c63a79464000a72d83c1cd231234
I20260806 16:06:22.845726 10954 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e45c63a79464000a72d83c1cd231234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:22 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:50826
I20260806 16:06:22.845722 11149 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e45c63a79464000a72d83c1cd231234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:22 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:45738
I20260806 16:06:22.845839 11334 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e45c63a79464000a72d83c1cd231234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:22 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:33928
I20260806 16:06:22.846285 11628 tablet_replica.cc:330] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a: stopping tablet replica
I20260806 16:06:22.846303 11629 tablet_replica.cc:330] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474: stopping tablet replica
I20260806 16:06:22.846370 11630 tablet_replica.cc:330] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8: stopping tablet replica
I20260806 16:06:22.846468 11629 raft_consensus.cc:2229] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:22.846470 11628 raft_consensus.cc:2229] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:22.846501 11630 raft_consensus.cc:2229] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:22.846638 11628 raft_consensus.cc:2258] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:22.846640 11629 raft_consensus.cc:2258] T 3e45c63a79464000a72d83c1cd231234 P 047cfb4f2a9e4d088b46199e48bc2474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:22.846764 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 10869
I20260806 16:06:22.854171 11628 ts_tablet_manager.cc:1905] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:22.859107 11628 ts_tablet_manager.cc:1918] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 16:06:22.859318 11628 log.cc:1196] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/3e45c63a79464000a72d83c1cd231234
I20260806 16:06:22.859566 11630 raft_consensus.cc:2258] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:22.859691 11628 ts_tablet_manager.cc:1939] T 3e45c63a79464000a72d83c1cd231234 P 598defb7de294eaa8020b041a87c5a1a: Deleting consensus metadata
I20260806 16:06:22.860572 11630 ts_tablet_manager.cc:1905] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:22.860968 10755 catalog_manager.cc:4841] TS 598defb7de294eaa8020b041a87c5a1a (127.8.49.2:41799): tablet 3e45c63a79464000a72d83c1cd231234 (table test-table [id=ec32b2d1b6d049ab8b7884ae5bf7b2a0]) successfully deleted
I20260806 16:06:22.864619 11630 ts_tablet_manager.cc:1918] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 16:06:22.864758 11630 log.cc:1196] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/3e45c63a79464000a72d83c1cd231234
I20260806 16:06:22.865144 11630 ts_tablet_manager.cc:1939] T 3e45c63a79464000a72d83c1cd231234 P 98ea9449706a41d89b32b412374c0df8: Deleting consensus metadata
I20260806 16:06:22.866575 10758 catalog_manager.cc:4841] TS 98ea9449706a41d89b32b412374c0df8 (127.8.49.3:44741): tablet 3e45c63a79464000a72d83c1cd231234 (table test-table [id=ec32b2d1b6d049ab8b7884ae5bf7b2a0]) successfully deleted
W20260806 16:06:22.870249 10758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:38821 (error 108)
I20260806 16:06:22.870561 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 11062
I20260806 16:06:22.874105 11633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:22.873523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:38821 (local address 127.0.0.1:54680)
0806 16:06:22.873884 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:22.873983 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:38821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":149,"threads_started":1}
W20260806 16:06:22.874281 10758 catalog_manager.cc:4584] TS 047cfb4f2a9e4d088b46199e48bc2474 (127.8.49.1:38821): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e45c63a79464000a72d83c1cd231234: Network error: Client connection negotiation failed: client connection to 127.8.49.1:38821: connect: Connection refused (error 111)
I20260806 16:06:22.888623 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 11256
I20260806 16:06:22.919034 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 10738
2026-08-06T16:06:22Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7814 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:23 dist-test-slave-7z32 krb5kdc[11641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:23 dist-test-slave-7z32 krb5kdc[11641](info): set up 2 sockets
Aug 06 16:06:23 dist-test-slave-7z32 krb5kdc[11641](info): commencing operation
krb5kdc: starting...
W20260806 16:06:24.278806 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.010s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:24 dist-test-slave-7z32 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:06:24.460950 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:37175
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:37175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:24.597057 11656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:24.597681 11656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:24.597743 11656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:24.603369 11656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:24.603492 11656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:24.603515 11656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:24.603542 11656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:24.603560 11656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:24.609758 11656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:37175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:37175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:24.611146 11656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:24.612327 11656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:06:24.624280 11656 server_base.cc:1034] running on GCE node
W20260806 16:06:24.624178 11665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:24.624250 11668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:24.624178 11671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:24.625612 11656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:24.627568 11656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:24.628789 11656 hybrid_clock.cc:648] HybridClock initialized: now 1786032384628741 us; error 66 us; skew 500 ppm
Aug 06 16:06:24 dist-test-slave-7z32 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032384, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:24.633177 11656 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:06:24.634680 11656 webserver.cc:460] Webserver started at http://127.8.49.62:42913/ using document root <none> and password file <none>
I20260806 16:06:24.635826 11656 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:24.635965 11656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:24.636363 11656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:24.639544 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "82c4d66e7aa44cbb902fe6cc6898952d"
format_stamp: "Formatted at 2026-08-06 16:06:24 on dist-test-slave-7z32"
server_key: "7fe94c54d929f13d4416cb69348702a8"
server_key_iv: "884ca1143cd9afc37b1f345ce842a8fe"
server_key_version: "encryptionkey@0"
I20260806 16:06:24.640367 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "82c4d66e7aa44cbb902fe6cc6898952d"
format_stamp: "Formatted at 2026-08-06 16:06:24 on dist-test-slave-7z32"
server_key: "7fe94c54d929f13d4416cb69348702a8"
server_key_iv: "884ca1143cd9afc37b1f345ce842a8fe"
server_key_version: "encryptionkey@0"
I20260806 16:06:24.647019 11656 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:06:24.652319 11681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:24.654547 11656 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:06:24.654816 11656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "82c4d66e7aa44cbb902fe6cc6898952d"
format_stamp: "Formatted at 2026-08-06 16:06:24 on dist-test-slave-7z32"
server_key: "7fe94c54d929f13d4416cb69348702a8"
server_key_iv: "884ca1143cd9afc37b1f345ce842a8fe"
server_key_version: "encryptionkey@0"
I20260806 16:06:24.655068 11656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:24.669485 11656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:24.675160 11656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:24.675493 11656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:24.685899 11656 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:37175
I20260806 16:06:24.685956 11746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:37175 every 8 connection(s)
I20260806 16:06:24.687525 11656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:24.691469 11747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:24.693281 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 11656
I20260806 16:06:24.693477 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:24.693853 8388 external_mini_cluster.cc:1428] Setting key 55c3667ef303db176e3ce1431ead2882
I20260806 16:06:24.700734 11747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d: Bootstrap starting.
I20260806 16:06:24.704180 11747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:24.705278 11747 log.cc:824] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:06:24 dist-test-slave-7z32 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:24.708760 11747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d: No bootstrap required, opened a new log
I20260806 16:06:24.713781 11747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4d66e7aa44cbb902fe6cc6898952d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37175 } }
I20260806 16:06:24.714174 11747 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:24.714277 11747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c4d66e7aa44cbb902fe6cc6898952d, State: Initialized, Role: FOLLOWER
I20260806 16:06:24.715184 11747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4d66e7aa44cbb902fe6cc6898952d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37175 } }
I20260806 16:06:24.715423 11747 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:24.715548 11747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:24.715719 11747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:24.716464 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:24.696401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.8.49.62:37175)
0806 16:06:24.697109 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:24.697134 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:24.697192 (+ 58us) server_negotiation.cc:407] Connection header received
0806 16:06:24.698231 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:24.698263 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:24.698934 (+ 671us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:24.699430 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:24.700749 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:24.702428 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:24.703787 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:24.704656 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:24.704870 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:24.709043 (+ 4173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:24.709052 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:24.709079 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:24.709125 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:24.712144 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:24.712816 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:24.712824 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:24.712838 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:24.712997 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:24.713335 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:24.713338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:24.713340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:24.713961 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:24.714257 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:24.714667 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 16:06:24.715124 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":198,"threads_started":1}
I20260806 16:06:24.717125 11747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4d66e7aa44cbb902fe6cc6898952d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37175 } }
I20260806 16:06:24.717600 11747 leader_election.cc:302] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82c4d66e7aa44cbb902fe6cc6898952d; no voters:
I20260806 16:06:24.718058 11747 leader_election.cc:288] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:24.718176 11753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:24.719103 11753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [term 1 LEADER]: Becoming Leader. State: Replica: 82c4d66e7aa44cbb902fe6cc6898952d, State: Running, Role: LEADER
I20260806 16:06:24.719805 11747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:24.719923 11753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4d66e7aa44cbb902fe6cc6898952d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37175 } }
I20260806 16:06:24.722551 11754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82c4d66e7aa44cbb902fe6cc6898952d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4d66e7aa44cbb902fe6cc6898952d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37175 } } }
I20260806 16:06:24.722748 11754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:24.722632 11755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82c4d66e7aa44cbb902fe6cc6898952d. Latest consensus state: current_term: 1 leader_uuid: "82c4d66e7aa44cbb902fe6cc6898952d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4d66e7aa44cbb902fe6cc6898952d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37175 } } }
I20260806 16:06:24.723639 11755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d [sys.catalog]: This master's current role is: LEADER
W20260806 16:06:24.726631 11777 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:06:24.726771 11777 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:06:24.726881 11778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:24.727638 11778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:24.735625 11778 catalog_manager.cc:1347] Generated new cluster ID: 9ec3e4fb307c470f9eb30d0b76140a93
I20260806 16:06:24.735762 11778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:24.753089 11778 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:24.754765 11778 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:24.775068 11778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82c4d66e7aa44cbb902fe6cc6898952d: Generated new TSK 0
I20260806 16:06:24.776243 11778 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:06:24.825163 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37175
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:24.997018 11789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:24.997473 11789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:24.997535 11789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:25.003459 11789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:25.003600 11789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:25.003684 11789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:25.011354 11789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:25.013070 11789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:25.014952 11789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:25.029963 11796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:25.030059 11789 server_base.cc:1034] running on GCE node
W20260806 16:06:25.029929 11797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:25.029929 11800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:25.031224 11789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:25.032243 11789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:25.033569 11789 hybrid_clock.cc:648] HybridClock initialized: now 1786032385033502 us; error 91 us; skew 500 ppm
Aug 06 16:06:25 dist-test-slave-7z32 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032385, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:25.038362 11789 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:06:25.039983 11789 webserver.cc:460] Webserver started at http://127.8.49.1:44473/ using document root <none> and password file <none>
I20260806 16:06:25.041194 11789 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:25.041460 11789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:25.041769 11789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:25.045135 11789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "03b79cf72c744733b5b2c005a43d5ca3"
format_stamp: "Formatted at 2026-08-06 16:06:25 on dist-test-slave-7z32"
server_key: "7b5af752a9650b51d0d0c0a73fde693b"
server_key_iv: "abb6581f03bf9f7f41735297471125ee"
server_key_version: "encryptionkey@0"
I20260806 16:06:25.046201 11789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "03b79cf72c744733b5b2c005a43d5ca3"
format_stamp: "Formatted at 2026-08-06 16:06:25 on dist-test-slave-7z32"
server_key: "7b5af752a9650b51d0d0c0a73fde693b"
server_key_iv: "abb6581f03bf9f7f41735297471125ee"
server_key_version: "encryptionkey@0"
I20260806 16:06:25.052886 11789 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 16:06:25.062242 11810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:25.064808 11789 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260806 16:06:25.065061 11789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "03b79cf72c744733b5b2c005a43d5ca3"
format_stamp: "Formatted at 2026-08-06 16:06:25 on dist-test-slave-7z32"
server_key: "7b5af752a9650b51d0d0c0a73fde693b"
server_key_iv: "abb6581f03bf9f7f41735297471125ee"
server_key_version: "encryptionkey@0"
I20260806 16:06:25.065245 11789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:25.091255 11789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:25.095952 11789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:25.096197 11789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:25.097023 11789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:25.098168 11789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:25.098227 11789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:25.098278 11789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:25.098294 11789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:25.114218 11954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:45469 every 8 connection(s)
I20260806 16:06:25.114287 11789 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:45469
I20260806 16:06:25.115998 11789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:25.119475 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 11789
I20260806 16:06:25.119639 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:25.120035 8388 external_mini_cluster.cc:1428] Setting key 5170dd78834f217bfafaea8d15f44311
Aug 06 16:06:25 dist-test-slave-7z32 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032385, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:06:25.137951 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:25.118752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:48197 (local address 127.8.49.62:37175)
0806 16:06:25.118954 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:25.118959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:25.120969 (+ 2010us) server_negotiation.cc:407] Connection header received
0806 16:06:25.123256 (+ 2287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:25.123263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:25.123355 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:25.123481 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:25.124921 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.125733 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.127675 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.127951 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.128058 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:25.132223 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:25.132228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:25.132230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:25.132269 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:25.134563 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:25.135405 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:25.135416 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:25.135418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:25.135591 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:25.136244 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:25.136247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:25.136248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:25.136513 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:25.136604 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:25.137345 (+ 741us) server_negotiation.cc:299] Negotiation successful
0806 16:06:25.137690 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 16:06:25.138899 11958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:25.119684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37175 (local address 127.8.49.1:48197)
0806 16:06:25.120753 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:25.120844 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:25.122962 (+ 2118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:25.123717 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:25.123724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:25.124159 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:25.124726 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:25.124740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.125902 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.125909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:25.127505 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:25.127523 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.128155 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.128162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:25.128432 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:25.129279 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:25.129309 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:25.131915 (+ 2606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:25.134770 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:25.134784 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:25.134807 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:25.135266 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:25.135779 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:25.135784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:25.135786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:25.136080 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:25.136636 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:25.136644 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:25.137010 (+ 366us) client_negotiation.cc:769] Sending connection context
0806 16:06:25.137213 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 16:06:25.137604 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":251,"threads_started":1}
I20260806 16:06:25.140981 11956 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37175
I20260806 16:06:25.141384 11956 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:25.142179 11956 heartbeater.cc:511] Master 127.8.49.62:37175 requested a full tablet report, sending...
I20260806 16:06:25.144573 11706 ts_manager.cc:194] Registered new tserver with Master: 03b79cf72c744733b5b2c005a43d5ca3 (127.8.49.1:45469)
I20260806 16:06:25.146505 11706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:48197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:06:25.164665 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37175
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:25.302146 11961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:25.302484 11961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:25.302529 11961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:25.306641 11961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:25.306721 11961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:25.306782 11961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:25.312817 11961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:25.314497 11961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:25.315757 11961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:25.328867 11973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:25.328866 11968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:25.329133 11961 server_base.cc:1034] running on GCE node
W20260806 16:06:25.328894 11971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:25.329739 11961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:25.330925 11961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:25.332239 11961 hybrid_clock.cc:648] HybridClock initialized: now 1786032385332225 us; error 50 us; skew 500 ppm
Aug 06 16:06:25 dist-test-slave-7z32 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032385, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:25.336062 11961 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:06:25.337306 11961 webserver.cc:460] Webserver started at http://127.8.49.2:36725/ using document root <none> and password file <none>
I20260806 16:06:25.337999 11961 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:25.338074 11961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:25.338322 11961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:25.340839 11961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "c32bff26d2604d9c8793b28e0abd9eed"
format_stamp: "Formatted at 2026-08-06 16:06:25 on dist-test-slave-7z32"
server_key: "d0b7f4a0afa28f9005e64fa8be41506a"
server_key_iv: "4867cd9a08e606e93230ce12e2055ccf"
server_key_version: "encryptionkey@0"
I20260806 16:06:25.341594 11961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "c32bff26d2604d9c8793b28e0abd9eed"
format_stamp: "Formatted at 2026-08-06 16:06:25 on dist-test-slave-7z32"
server_key: "d0b7f4a0afa28f9005e64fa8be41506a"
server_key_iv: "4867cd9a08e606e93230ce12e2055ccf"
server_key_version: "encryptionkey@0"
I20260806 16:06:25.347147 11961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260806 16:06:25.350567 11983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:25.352504 11961 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 16:06:25.352656 11961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "c32bff26d2604d9c8793b28e0abd9eed"
format_stamp: "Formatted at 2026-08-06 16:06:25 on dist-test-slave-7z32"
server_key: "d0b7f4a0afa28f9005e64fa8be41506a"
server_key_iv: "4867cd9a08e606e93230ce12e2055ccf"
server_key_version: "encryptionkey@0"
I20260806 16:06:25.352828 11961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:25.388365 11961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:25.392977 11961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:25.393258 11961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:25.394111 11961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:25.395777 11961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:25.395875 11961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:25.395928 11961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:25.395951 11961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:25.410301 11961 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:43649
I20260806 16:06:25.410413 12130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:43649 every 8 connection(s)
I20260806 16:06:25.411595 11961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:25.417528 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 11961
I20260806 16:06:25.417692 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:25.418090 8388 external_mini_cluster.cc:1428] Setting key fa9dde8a8588a5ba2fcc6582946b7a40
Aug 06 16:06:25 dist-test-slave-7z32 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032385, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:06:25.435001 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:25.414568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:40095 (local address 127.8.49.62:37175)
0806 16:06:25.414801 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:25.414807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:25.416073 (+ 1266us) server_negotiation.cc:407] Connection header received
0806 16:06:25.418107 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:25.418111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:25.418219 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:25.418359 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:25.420539 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.421453 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.423767 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.424248 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.424425 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:25.428573 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:25.428583 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:25.428589 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:25.428640 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:25.431001 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:25.432021 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:25.432030 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:25.432034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:25.432156 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:25.432874 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:25.432880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:25.432881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:25.433189 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:25.433350 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:25.434266 (+ 916us) server_negotiation.cc:299] Negotiation successful
0806 16:06:25.434485 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:06:25.436415 12134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:25.415121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37175 (local address 127.8.49.2:40095)
0806 16:06:25.415823 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:25.415894 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:25.417737 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:25.418673 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:25.418690 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:25.419419 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:25.420298 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:25.420331 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.421659 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.421667 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:25.423562 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:25.423585 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.424531 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.424535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:25.424860 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:25.425818 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:25.425841 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:25.428223 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:25.431270 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:25.431281 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:25.431306 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:25.431822 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:25.432318 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:25.432322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:25.432324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:25.432549 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:25.433346 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:25.433357 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:25.433918 (+ 561us) client_negotiation.cc:769] Sending connection context
0806 16:06:25.434197 (+ 279us) client_negotiation.cc:240] Negotiation successful
0806 16:06:25.434957 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":181,"threads_started":1}
I20260806 16:06:25.438265 12131 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37175
I20260806 16:06:25.438692 12131 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:25.439494 12131 heartbeater.cc:511] Master 127.8.49.62:37175 requested a full tablet report, sending...
I20260806 16:06:25.441131 11706 ts_manager.cc:194] Registered new tserver with Master: c32bff26d2604d9c8793b28e0abd9eed (127.8.49.2:43649)
I20260806 16:06:25.442188 11706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:40095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:06:25.460405 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37175
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:25.625254 12142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:25.625700 12142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:25.625765 12142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:25.631153 12142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:25.631634 12142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:25.631857 12142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:25.639375 12142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:25.641222 12142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:25.642784 12142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:06:25.658572 12142 server_base.cc:1034] running on GCE node
W20260806 16:06:25.658488 12148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:25.658504 12151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:25.658488 12149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:25.659907 12142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:25.661032 12142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:25.662329 12142 hybrid_clock.cc:648] HybridClock initialized: now 1786032385662288 us; error 60 us; skew 500 ppm
Aug 06 16:06:25 dist-test-slave-7z32 krb5kdc[11641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032385, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:25.666678 12142 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:06:25.668697 12142 webserver.cc:460] Webserver started at http://127.8.49.3:46575/ using document root <none> and password file <none>
I20260806 16:06:25.669515 12142 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:25.669994 12142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:25.670490 12142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:25.673998 12142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "2c0ec532da6248758f150c78636826a5"
format_stamp: "Formatted at 2026-08-06 16:06:25 on dist-test-slave-7z32"
server_key: "d1395a66e8a7546952c2cc185c3422f5"
server_key_iv: "96d107f8a78804c05f44758eae4025f7"
server_key_version: "encryptionkey@0"
I20260806 16:06:25.674942 12142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "2c0ec532da6248758f150c78636826a5"
format_stamp: "Formatted at 2026-08-06 16:06:25 on dist-test-slave-7z32"
server_key: "d1395a66e8a7546952c2cc185c3422f5"
server_key_iv: "96d107f8a78804c05f44758eae4025f7"
server_key_version: "encryptionkey@0"
I20260806 16:06:25.680936 12142 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 16:06:25.684886 12163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:25.687390 12142 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:25.687634 12142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "2c0ec532da6248758f150c78636826a5"
format_stamp: "Formatted at 2026-08-06 16:06:25 on dist-test-slave-7z32"
server_key: "d1395a66e8a7546952c2cc185c3422f5"
server_key_iv: "96d107f8a78804c05f44758eae4025f7"
server_key_version: "encryptionkey@0"
I20260806 16:06:25.687821 12142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:25.708813 12142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:25.714659 12142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:25.714995 12142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:25.715839 12142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:25.717411 12142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:25.717517 12142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:25.717885 12142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:25.718017 12142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:25.733291 12142 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:42123
I20260806 16:06:25.733471 12312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:42123 every 8 connection(s)
I20260806 16:06:25.734472 12142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:25.742903 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 12142
I20260806 16:06:25.743075 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:25.743438 8388 external_mini_cluster.cc:1428] Setting key fb13704cc28d7e4378e8e632761e08df
Aug 06 16:06:25 dist-test-slave-7z32 krb5kdc[11641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032385, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:25.754495 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:25.737864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:45509 (local address 127.8.49.62:37175)
0806 16:06:25.738349 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:25.738354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:25.739671 (+ 1317us) server_negotiation.cc:407] Connection header received
0806 16:06:25.741845 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:25.741849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:25.741921 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:25.742049 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:25.744003 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.744882 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.746467 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.746731 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.746900 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:25.750012 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:25.750016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:25.750018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:25.750048 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:25.751924 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:25.752649 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:25.752653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:25.752654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:25.752740 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:25.753169 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:25.753173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:25.753174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:25.753398 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:25.753531 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:25.754142 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 16:06:25.754298 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330}
I20260806 16:06:25.755581 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:25.738476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37175 (local address 127.8.49.3:45509)
0806 16:06:25.739425 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:25.739509 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:25.741531 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:25.742281 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:25.742290 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:25.742978 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:25.743792 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:25.743820 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.745067 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.745077 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:25.746310 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:25.746322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:25.746972 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:25.746976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:25.747230 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:25.748024 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:25.748043 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:25.749794 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:25.752088 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:25.752095 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:25.752146 (+ 51us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:25.752453 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:25.752853 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:25.752857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:25.752859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:25.753030 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:25.753532 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:25.753542 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:25.753851 (+ 309us) client_negotiation.cc:769] Sending connection context
0806 16:06:25.754083 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 16:06:25.754482 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":292,"threads_started":1}
I20260806 16:06:25.757683 12313 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37175
I20260806 16:06:25.758000 12313 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:25.758668 12313 heartbeater.cc:511] Master 127.8.49.62:37175 requested a full tablet report, sending...
I20260806 16:06:25.760156 11706 ts_manager.cc:194] Registered new tserver with Master: 2c0ec532da6248758f150c78636826a5 (127.8.49.3:42123)
I20260806 16:06:25.760859 11706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:45509
I20260806 16:06:25.773612 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 16:06:25.798662 11750 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:06:25.796297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42466 (local address 127.8.49.62:37175)
0806 16:06:25.796616 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:25.796621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:25.796647 (+ 26us) server_negotiation.cc:407] Connection header received
0806 16:06:25.796889 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:25.796894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:25.796964 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:25.797081 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:25.798464 (+ 1383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42466: BlockingRecv error: recv got EOF from 127.0.0.1:42466 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
I20260806 16:06:25.803100 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 11789
I20260806 16:06:25.842268 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 11961
I20260806 16:06:25.865746 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 12142
I20260806 16:06:25.881071 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 11656
2026-08-06T16:06:25Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2959 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:25 dist-test-slave-7z32 krb5kdc[12327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:25 dist-test-slave-7z32 krb5kdc[12327](info): set up 2 sockets
Aug 06 16:06:25 dist-test-slave-7z32 krb5kdc[12327](info): commencing operation
krb5kdc: starting...
W20260806 16:06:27.254040 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:27 dist-test-slave-7z32 krb5kdc[12327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:06:27.431169 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42537
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:42537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:27.584726 12342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:27.585187 12342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:27.585238 12342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:27.590513 12342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:27.590641 12342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:27.590665 12342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:27.590688 12342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:27.590705 12342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:27.599964 12342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:42537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:27.601668 12342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:27.603350 12342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:27.617125 12351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:27.617185 12352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:27.617357 12355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:27.617718 12342 server_base.cc:1034] running on GCE node
I20260806 16:06:27.618563 12342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:27.620249 12342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:27.623718 12342 hybrid_clock.cc:648] HybridClock initialized: now 1786032387623586 us; error 137 us; skew 500 ppm
Aug 06 16:06:27 dist-test-slave-7z32 krb5kdc[12327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032387, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:27.628541 12342 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:06:27.629951 12342 webserver.cc:460] Webserver started at http://127.8.49.62:39969/ using document root <none> and password file <none>
I20260806 16:06:27.630786 12342 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:27.630993 12342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:27.631503 12342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:27.634282 12342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "1d3956d396f74df6bec8ad8ed4352196"
format_stamp: "Formatted at 2026-08-06 16:06:27 on dist-test-slave-7z32"
server_key: "39aec102a0316343498ada02015e1f53"
server_key_iv: "0ddd62b1adb333a07a5f4195b543e245"
server_key_version: "encryptionkey@0"
I20260806 16:06:27.635113 12342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "1d3956d396f74df6bec8ad8ed4352196"
format_stamp: "Formatted at 2026-08-06 16:06:27 on dist-test-slave-7z32"
server_key: "39aec102a0316343498ada02015e1f53"
server_key_iv: "0ddd62b1adb333a07a5f4195b543e245"
server_key_version: "encryptionkey@0"
I20260806 16:06:27.640489 12342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:06:27.644394 12362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:27.646440 12342 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:06:27.646688 12342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "1d3956d396f74df6bec8ad8ed4352196"
format_stamp: "Formatted at 2026-08-06 16:06:27 on dist-test-slave-7z32"
server_key: "39aec102a0316343498ada02015e1f53"
server_key_iv: "0ddd62b1adb333a07a5f4195b543e245"
server_key_version: "encryptionkey@0"
I20260806 16:06:27.646890 12342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:27.660969 12342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:27.665726 12342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:27.666011 12342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:27.676121 12342 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:42537
I20260806 16:06:27.676469 12429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:42537 every 8 connection(s)
I20260806 16:06:27.677475 12342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:27.682125 12430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:27.683866 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 12342
I20260806 16:06:27.684075 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:27.684401 8388 external_mini_cluster.cc:1428] Setting key 1384eb288a1b496963a0f0282b743579
I20260806 16:06:27.693068 12430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196: Bootstrap starting.
Aug 06 16:06:27 dist-test-slave-7z32 krb5kdc[12327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:27.697352 12430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:27.698437 12430 log.cc:824] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:27.701843 12430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196: No bootstrap required, opened a new log
I20260806 16:06:27.704280 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:27.686431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.8.49.62:42537)
0806 16:06:27.687114 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:27.687136 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:27.687183 (+ 47us) server_negotiation.cc:407] Connection header received
0806 16:06:27.688121 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:27.688144 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:27.688608 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:27.689000 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:27.690393 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:27.691796 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:27.693307 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:27.694045 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:27.694229 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:27.698121 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:27.698129 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:27.698152 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:27.698199 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:27.700572 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:27.701187 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:27.701193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:27.701200 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:27.701340 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:27.701690 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:27.701693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:27.701694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:27.702307 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:27.702534 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:27.702960 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 16:06:27.703329 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":202,"threads_started":1}
I20260806 16:06:27.708225 12430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d3956d396f74df6bec8ad8ed4352196" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42537 } }
I20260806 16:06:27.708637 12430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:27.708753 12430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d3956d396f74df6bec8ad8ed4352196, State: Initialized, Role: FOLLOWER
I20260806 16:06:27.709263 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:06:27.709461 12430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d3956d396f74df6bec8ad8ed4352196" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42537 } }
I20260806 16:06:27.709668 12430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:27.709748 12430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:27.709867 12430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:27.712034 12430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d3956d396f74df6bec8ad8ed4352196" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42537 } }
I20260806 16:06:27.712664 12430 leader_election.cc:302] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d3956d396f74df6bec8ad8ed4352196; no voters:
I20260806 16:06:27.713315 12430 leader_election.cc:288] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:27.713429 12440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:27.713727 12440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [term 1 LEADER]: Becoming Leader. State: Replica: 1d3956d396f74df6bec8ad8ed4352196, State: Running, Role: LEADER
I20260806 16:06:27.714241 12440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d3956d396f74df6bec8ad8ed4352196" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42537 } }
I20260806 16:06:27.715298 12430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:27.716025 12440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d3956d396f74df6bec8ad8ed4352196. Latest consensus state: current_term: 1 leader_uuid: "1d3956d396f74df6bec8ad8ed4352196" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d3956d396f74df6bec8ad8ed4352196" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42537 } } }
I20260806 16:06:27.716177 12440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:27.716686 12444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:27.716578 12441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d3956d396f74df6bec8ad8ed4352196" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d3956d396f74df6bec8ad8ed4352196" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42537 } } }
I20260806 16:06:27.717010 12441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:27.721365 12444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:27.729907 12444 catalog_manager.cc:1347] Generated new cluster ID: 981cb2f86ea8457d8b36b674f812ee00
I20260806 16:06:27.730038 12444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:27.737582 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:27.731556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38770 (local address 127.8.49.62:42537)
0806 16:06:27.731731 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:27.731736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:27.731992 (+ 256us) server_negotiation.cc:407] Connection header received
0806 16:06:27.732240 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:27.732244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:27.732310 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:27.732377 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:27.733113 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:27.733700 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:27.734642 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:27.735009 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:27.735139 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:27.736198 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:27.736203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:27.736205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:27.736773 (+ 568us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:27.736812 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:27.736831 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:27.736960 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:27.737266 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 16:06:27.737361 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 16:06:27.750523 12444 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:27.753504 12444 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:27.774930 12444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d3956d396f74df6bec8ad8ed4352196: Generated new TSK 0
I20260806 16:06:27.775861 12444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:06:27.787020 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 12342
2026-08-06T16:06:27Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1902 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:27 dist-test-slave-7z32 krb5kdc[12477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:27 dist-test-slave-7z32 krb5kdc[12477](info): set up 2 sockets
Aug 06 16:06:27 dist-test-slave-7z32 krb5kdc[12477](info): commencing operation
krb5kdc: starting...
W20260806 16:06:29.144915 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:29 dist-test-slave-7z32 krb5kdc[12477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:06:29.332840 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:40817
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:40817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:29.471939 12494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:29.472293 12494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:29.472371 12494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:29.476680 12494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:29.476799 12494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:29.476833 12494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:29.476864 12494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:29.476923 12494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:29.482724 12494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:40817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:40817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:29.484104 12494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:29.485275 12494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:29.496658 12503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:29.496685 12501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:29.496873 12499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:29.497637 12494 server_base.cc:1034] running on GCE node
I20260806 16:06:29.498363 12494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:29.499876 12494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:29.501271 12494 hybrid_clock.cc:648] HybridClock initialized: now 1786032389501195 us; error 105 us; skew 500 ppm
Aug 06 16:06:29 dist-test-slave-7z32 krb5kdc[12477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032389, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:29.504833 12494 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:06:29.506102 12494 webserver.cc:460] Webserver started at http://127.8.49.62:44225/ using document root <none> and password file <none>
I20260806 16:06:29.506776 12494 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:29.506912 12494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:29.507215 12494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:29.509562 12494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "d6a41c92349c4f0b8bf1af0ef7e3642b"
format_stamp: "Formatted at 2026-08-06 16:06:29 on dist-test-slave-7z32"
server_key: "42a86cee80b247b0bbd8c9f324373e5f"
server_key_iv: "7fae6dc00b887e550f570e043086be08"
server_key_version: "encryptionkey@0"
I20260806 16:06:29.510242 12494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "d6a41c92349c4f0b8bf1af0ef7e3642b"
format_stamp: "Formatted at 2026-08-06 16:06:29 on dist-test-slave-7z32"
server_key: "42a86cee80b247b0bbd8c9f324373e5f"
server_key_iv: "7fae6dc00b887e550f570e043086be08"
server_key_version: "encryptionkey@0"
I20260806 16:06:29.515312 12494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260806 16:06:29.518589 12511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:29.520843 12494 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:29.521103 12494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "d6a41c92349c4f0b8bf1af0ef7e3642b"
format_stamp: "Formatted at 2026-08-06 16:06:29 on dist-test-slave-7z32"
server_key: "42a86cee80b247b0bbd8c9f324373e5f"
server_key_iv: "7fae6dc00b887e550f570e043086be08"
server_key_version: "encryptionkey@0"
I20260806 16:06:29.521276 12494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:29.550225 12494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:29.555130 12494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:29.555424 12494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:29.566370 12494 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:40817
I20260806 16:06:29.566406 12579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:40817 every 8 connection(s)
I20260806 16:06:29.567772 12494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:29.571425 12580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:29.575599 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 12494
I20260806 16:06:29.575764 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:29.576105 8388 external_mini_cluster.cc:1428] Setting key 688246c4aa986d9a91f2e3d90e1d1475
I20260806 16:06:29.578547 12580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b: Bootstrap starting.
I20260806 16:06:29.581502 12580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:29.582549 12580 log.cc:824] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:29.585736 12580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b: No bootstrap required, opened a new log
Aug 06 16:06:29 dist-test-slave-7z32 krb5kdc[12477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:29.591580 12580 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a41c92349c4f0b8bf1af0ef7e3642b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40817 } }
I20260806 16:06:29.591995 12580 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:29.592082 12580 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a41c92349c4f0b8bf1af0ef7e3642b, State: Initialized, Role: FOLLOWER
I20260806 16:06:29.592842 12580 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a41c92349c4f0b8bf1af0ef7e3642b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40817 } }
I20260806 16:06:29.593150 12580 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:29.593284 12580 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:29.593417 12580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:29.594910 12580 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a41c92349c4f0b8bf1af0ef7e3642b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40817 } }
I20260806 16:06:29.595381 12580 leader_election.cc:302] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6a41c92349c4f0b8bf1af0ef7e3642b; no voters:
I20260806 16:06:29.595810 12580 leader_election.cc:288] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:29.596068 12586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:29.596549 12586 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [term 1 LEADER]: Becoming Leader. State: Replica: d6a41c92349c4f0b8bf1af0ef7e3642b, State: Running, Role: LEADER
I20260806 16:06:29.597055 12586 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a41c92349c4f0b8bf1af0ef7e3642b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40817 } }
I20260806 16:06:29.597930 12580 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:29.600001 12589 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6a41c92349c4f0b8bf1af0ef7e3642b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a41c92349c4f0b8bf1af0ef7e3642b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40817 } } }
I20260806 16:06:29.600147 12583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:29.578296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41268 (local address 127.8.49.62:40817)
0806 16:06:29.579084 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:29.579107 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:29.579163 (+ 56us) server_negotiation.cc:407] Connection header received
0806 16:06:29.580129 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:29.580154 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:29.580656 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:29.581103 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:29.582427 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:29.583871 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:29.585570 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:29.586521 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:29.586774 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:29.590907 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:29.590919 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:29.590957 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:29.591014 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:29.594051 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:29.594896 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:29.594955 (+ 59us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:29.594963 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:29.595162 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:29.595741 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:29.595748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:29.595750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:29.596851 (+ 1101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:29.597115 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:29.598018 (+ 903us) server_negotiation.cc:299] Negotiation successful
0806 16:06:29.598543 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":254,"threads_started":1}
I20260806 16:06:29.600250 12589 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:29.600965 12605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:29.601169 12590 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6a41c92349c4f0b8bf1af0ef7e3642b. Latest consensus state: current_term: 1 leader_uuid: "d6a41c92349c4f0b8bf1af0ef7e3642b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6a41c92349c4f0b8bf1af0ef7e3642b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40817 } } }
I20260806 16:06:29.601528 12590 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:29.607151 12605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:29.617337 12605 catalog_manager.cc:1347] Generated new cluster ID: a79163ee37d8418ba74e61c0daefe117
I20260806 16:06:29.617578 12605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:29.634171 12605 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:29.635936 12605 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:29.652873 12605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6a41c92349c4f0b8bf1af0ef7e3642b: Generated new TSK 0
I20260806 16:06:29.654230 12605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:06:29.704815 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:40817
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:29.875941 12616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:29.876403 12616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:29.876461 12616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:29.881378 12616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:29.881518 12616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:29.881595 12616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:29.890048 12616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:40817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:29.891825 12616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:29.893355 12616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:29.908344 12623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:29.908535 12624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:29.908721 12616 server_base.cc:1034] running on GCE node
W20260806 16:06:29.908362 12626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:29.909569 12616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:29.910413 12616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:29.911672 12616 hybrid_clock.cc:648] HybridClock initialized: now 1786032389911582 us; error 88 us; skew 500 ppm
Aug 06 16:06:29 dist-test-slave-7z32 krb5kdc[12477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032389, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:29.916198 12616 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:06:29.917402 12616 webserver.cc:460] Webserver started at http://127.8.49.1:45925/ using document root <none> and password file <none>
I20260806 16:06:29.918134 12616 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:29.918246 12616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:29.918548 12616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:29.921176 12616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "935600deb4524d4c923979401729d402"
format_stamp: "Formatted at 2026-08-06 16:06:29 on dist-test-slave-7z32"
server_key: "5523b33b6dec3ebdc79178123270757d"
server_key_iv: "8faf283b3da5178ab7bf4cf7a7b288e4"
server_key_version: "encryptionkey@0"
I20260806 16:06:29.921855 12616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "935600deb4524d4c923979401729d402"
format_stamp: "Formatted at 2026-08-06 16:06:29 on dist-test-slave-7z32"
server_key: "5523b33b6dec3ebdc79178123270757d"
server_key_iv: "8faf283b3da5178ab7bf4cf7a7b288e4"
server_key_version: "encryptionkey@0"
I20260806 16:06:29.927081 12616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 16:06:29.930383 12634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:29.932366 12616 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:29.932598 12616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "935600deb4524d4c923979401729d402"
format_stamp: "Formatted at 2026-08-06 16:06:29 on dist-test-slave-7z32"
server_key: "5523b33b6dec3ebdc79178123270757d"
server_key_iv: "8faf283b3da5178ab7bf4cf7a7b288e4"
server_key_version: "encryptionkey@0"
I20260806 16:06:29.932778 12616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:29.949771 12616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:29.953588 12616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:29.953822 12616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:29.954545 12616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:29.955653 12616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:29.955735 12616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:29.955821 12616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:29.955870 12616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:29.968137 12616 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:43709
I20260806 16:06:29.968384 12783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:43709 every 8 connection(s)
I20260806 16:06:29.969414 12616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:29.978798 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 12616
I20260806 16:06:29.979048 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:29.979394 8388 external_mini_cluster.cc:1428] Setting key 7f09991147c61497edbb5238185a5f57
Aug 06 16:06:29 dist-test-slave-7z32 krb5kdc[12477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032389, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:29.992548 12583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:29.972675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:43781 (local address 127.8.49.62:40817)
0806 16:06:29.972923 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:29.972928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:29.974043 (+ 1115us) server_negotiation.cc:407] Connection header received
0806 16:06:29.976139 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:29.976144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:29.976239 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:29.976384 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:29.978246 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:29.979113 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:29.981254 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:29.981543 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:29.981948 (+ 405us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:29.985926 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:29.985934 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:29.985939 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:29.985985 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:29.988599 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:29.989607 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:29.989615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:29.989621 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:29.989793 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:29.990477 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:29.990485 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:29.990489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:29.990894 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:29.991086 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:29.991817 (+ 731us) server_negotiation.cc:299] Negotiation successful
0806 16:06:29.992047 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260806 16:06:29.993285 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:29.973019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:40817 (local address 127.8.49.1:43781)
0806 16:06:29.973790 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:29.973861 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:29.975795 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:29.976666 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:29.976676 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:29.977255 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:29.978002 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:29.978021 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:29.979320 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:29.979325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:29.981020 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:29.981037 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:29.982027 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:29.982032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:29.982298 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:29.983237 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:29.983259 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:29.985579 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:29.988929 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:29.988941 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:29.988961 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:29.989416 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:29.989952 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:29.989954 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:29.989956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:29.990308 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:29.991114 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:29.991121 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:29.991470 (+ 349us) client_negotiation.cc:769] Sending connection context
0806 16:06:29.991701 (+ 231us) client_negotiation.cc:240] Negotiation successful
0806 16:06:29.992107 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":140,"threads_started":1}
I20260806 16:06:29.995235 12784 heartbeater.cc:347] Connected to a master server at 127.8.49.62:40817
I20260806 16:06:29.995895 12784 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
I20260806 16:06:29.997095 12784 heartbeater.cc:511] Master 127.8.49.62:40817 requested a full tablet report, sending...
I20260806 16:06:30.000002 12537 ts_manager.cc:194] Registered new tserver with Master: 935600deb4524d4c923979401729d402 (127.8.49.1:43709)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:06:30.002152 12537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:43781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:06:30.028133 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:40817
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:30.191728 12789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:30.192162 12789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:30.192219 12789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:30.197351 12789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:30.197515 12789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:30.197634 12789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:30.205066 12789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:40817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:30.206784 12789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:30.208349 12789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:30.222185 12795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:30.222213 12796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:30.222189 12801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:30.222672 12789 server_base.cc:1034] running on GCE node
I20260806 16:06:30.223563 12789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:30.224360 12789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:30.227774 12789 hybrid_clock.cc:648] HybridClock initialized: now 1786032390227710 us; error 81 us; skew 500 ppm
Aug 06 16:06:30 dist-test-slave-7z32 krb5kdc[12477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032390, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:30.232133 12789 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:06:30.233417 12789 webserver.cc:460] Webserver started at http://127.8.49.2:39515/ using document root <none> and password file <none>
I20260806 16:06:30.234179 12789 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:30.234290 12789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:30.234565 12789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:30.237089 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "d9b580584e7648ddb061d25c403413f0"
format_stamp: "Formatted at 2026-08-06 16:06:30 on dist-test-slave-7z32"
server_key: "66e9b12fae0c0cfb67cf90c51f0ef37d"
server_key_iv: "81f46198d6ab017c30edc9031c5c0a03"
server_key_version: "encryptionkey@0"
I20260806 16:06:30.237779 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "d9b580584e7648ddb061d25c403413f0"
format_stamp: "Formatted at 2026-08-06 16:06:30 on dist-test-slave-7z32"
server_key: "66e9b12fae0c0cfb67cf90c51f0ef37d"
server_key_iv: "81f46198d6ab017c30edc9031c5c0a03"
server_key_version: "encryptionkey@0"
I20260806 16:06:30.243559 12789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 16:06:30.247042 12809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:30.248991 12789 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:30.249215 12789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "d9b580584e7648ddb061d25c403413f0"
format_stamp: "Formatted at 2026-08-06 16:06:30 on dist-test-slave-7z32"
server_key: "66e9b12fae0c0cfb67cf90c51f0ef37d"
server_key_iv: "81f46198d6ab017c30edc9031c5c0a03"
server_key_version: "encryptionkey@0"
I20260806 16:06:30.249378 12789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:30.275470 12789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:30.280488 12789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:30.280787 12789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:30.281581 12789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:30.282652 12789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:30.282742 12789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:30.282862 12789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:30.282928 12789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 16:06:30.296720 12789 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:41445
I20260806 16:06:30.296851 12950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:41445 every 8 connection(s)
I20260806 16:06:30.298921 12789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:30.301139 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 12789
I20260806 16:06:30.301563 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:30.302140 8388 external_mini_cluster.cc:1428] Setting key 4cc39b05842626d14de5baef3524d957
Aug 06 16:06:30 dist-test-slave-7z32 krb5kdc[12477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032390, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:06:30.321648 12583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:30.302122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:54809 (local address 127.8.49.62:40817)
0806 16:06:30.302407 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:30.302415 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:30.304074 (+ 1659us) server_negotiation.cc:407] Connection header received
0806 16:06:30.306039 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:30.306044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:30.306131 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:30.306224 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:30.308127 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:30.308973 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:30.310630 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:30.311128 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:30.311308 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:30.314958 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:30.314968 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:30.314975 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:30.315034 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:30.317278 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:30.318264 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:30.318269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:30.318271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:30.318480 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:30.319296 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:30.319299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:30.319301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:30.319736 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:30.319862 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:30.321191 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0806 16:06:30.321371 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260806 16:06:30.323045 12953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:30.302594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:40817 (local address 127.8.49.2:54809)
0806 16:06:30.303590 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:30.303661 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:30.305630 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:30.306491 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:30.306502 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:30.307162 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:30.307914 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:30.307934 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:30.309232 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:30.309237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:30.310398 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:30.310417 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:30.311373 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:30.311376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:30.311589 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:30.312569 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:30.312592 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:30.314607 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:30.317511 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:30.317521 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:30.317541 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:30.317985 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:30.318738 (+ 753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:30.318742 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:30.318744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:30.319120 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:30.319927 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:30.319936 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:30.320576 (+ 640us) client_negotiation.cc:769] Sending connection context
0806 16:06:30.321004 (+ 428us) client_negotiation.cc:240] Negotiation successful
0806 16:06:30.321639 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":297,"threads_started":1}
I20260806 16:06:30.326495 12951 heartbeater.cc:347] Connected to a master server at 127.8.49.62:40817
I20260806 16:06:30.327031 12951 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:30.328145 12951 heartbeater.cc:511] Master 127.8.49.62:40817 requested a full tablet report, sending...
I20260806 16:06:30.330183 12537 ts_manager.cc:194] Registered new tserver with Master: d9b580584e7648ddb061d25c403413f0 (127.8.49.2:41445)
I20260806 16:06:30.331235 12537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:54809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:06:30.347554 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:40817
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:30.492774 12960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:30.493109 12960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:30.493156 12960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:30.497905 12960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:30.497987 12960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:30.498054 12960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:30.504180 12960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:40817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:30.505637 12960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:30.506939 12960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:30.518968 12969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:30.518978 12972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:30.518981 12970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:30.519373 12960 server_base.cc:1034] running on GCE node
I20260806 16:06:30.520001 12960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:30.520882 12960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:30.522123 12960 hybrid_clock.cc:648] HybridClock initialized: now 1786032390522082 us; error 56 us; skew 500 ppm
Aug 06 16:06:30 dist-test-slave-7z32 krb5kdc[12477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032390, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:30.525859 12960 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:06:30.527272 12960 webserver.cc:460] Webserver started at http://127.8.49.3:34397/ using document root <none> and password file <none>
I20260806 16:06:30.528200 12960 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:30.528327 12960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:30.528656 12960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:30.531291 12960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "b02a51b017d14acbbdb22a9d09f0f2ab"
format_stamp: "Formatted at 2026-08-06 16:06:30 on dist-test-slave-7z32"
server_key: "19995fb2e7446165e5a004a9040cbbbf"
server_key_iv: "7d61b28d18430f520b0f6ff472793c60"
server_key_version: "encryptionkey@0"
I20260806 16:06:30.531970 12960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "b02a51b017d14acbbdb22a9d09f0f2ab"
format_stamp: "Formatted at 2026-08-06 16:06:30 on dist-test-slave-7z32"
server_key: "19995fb2e7446165e5a004a9040cbbbf"
server_key_iv: "7d61b28d18430f520b0f6ff472793c60"
server_key_version: "encryptionkey@0"
I20260806 16:06:30.536885 12960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:06:30.540493 12980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:30.542268 12960 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:30.542469 12960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "b02a51b017d14acbbdb22a9d09f0f2ab"
format_stamp: "Formatted at 2026-08-06 16:06:30 on dist-test-slave-7z32"
server_key: "19995fb2e7446165e5a004a9040cbbbf"
server_key_iv: "7d61b28d18430f520b0f6ff472793c60"
server_key_version: "encryptionkey@0"
I20260806 16:06:30.542625 12960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:30.570593 12960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:30.576718 12960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:30.577699 12960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:30.578810 12960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:30.580020 12960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:30.580086 12960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:30.580142 12960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:30.580163 12960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:30.596203 12960 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:34813
I20260806 16:06:30.596243 13126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:34813 every 8 connection(s)
I20260806 16:06:30.597589 12960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:30.599601 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 12960
I20260806 16:06:30.599735 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:30.600231 8388 external_mini_cluster.cc:1428] Setting key 33b37598cd6e4b4fcf8a2e832e269195
Aug 06 16:06:30 dist-test-slave-7z32 krb5kdc[12477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032390, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:30.619746 12583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:30.600639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:47681 (local address 127.8.49.62:40817)
0806 16:06:30.600861 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:30.600868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:30.602172 (+ 1304us) server_negotiation.cc:407] Connection header received
0806 16:06:30.604773 (+ 2601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:30.604778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:30.604860 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:30.605082 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:30.607192 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:30.607812 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:30.609625 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:30.609942 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:30.610073 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:30.614228 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:30.614239 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:30.614243 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:30.614289 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:30.616134 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:30.617193 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:30.617197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:30.617199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:30.617331 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:30.618047 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:30.618050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:30.618051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:30.618392 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:30.618488 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:30.619445 (+ 957us) server_negotiation.cc:299] Negotiation successful
0806 16:06:30.619596 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 16:06:30.621270 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:30.601117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:40817 (local address 127.8.49.3:47681)
0806 16:06:30.601943 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:30.602018 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:30.604177 (+ 2159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:30.605390 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:30.605402 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:30.606005 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:30.606774 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:30.606793 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:30.608000 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:30.608004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:30.609411 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:30.609427 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:30.610202 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:30.610207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:30.610465 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:30.611552 (+ 1087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:30.611575 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:30.613735 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:30.616323 (+ 2588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:30.616331 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:30.616384 (+ 53us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:30.617013 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:30.617465 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:30.617469 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:30.617471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:30.617716 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:30.618557 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:30.618565 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:30.619097 (+ 532us) client_negotiation.cc:769] Sending connection context
0806 16:06:30.619335 (+ 238us) client_negotiation.cc:240] Negotiation successful
0806 16:06:30.619805 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":170,"threads_started":1}
I20260806 16:06:30.623411 13128 heartbeater.cc:347] Connected to a master server at 127.8.49.62:40817
I20260806 16:06:30.623980 13128 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:30.624739 13128 heartbeater.cc:511] Master 127.8.49.62:40817 requested a full tablet report, sending...
I20260806 16:06:30.626394 12537 ts_manager.cc:194] Registered new tserver with Master: b02a51b017d14acbbdb22a9d09f0f2ab (127.8.49.3:34813)
I20260806 16:06:30.627334 12537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:47681
I20260806 16:06:30.631139 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 16:06:30 dist-test-slave-7z32 krb5kdc[12477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032390, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 16:06:30 dist-test-slave-7z32 krb5kdc[12477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032390, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:30.671350 12583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:30.659307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41272 (local address 127.8.49.62:40817)
0806 16:06:30.659543 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:30.659549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:30.659710 (+ 161us) server_negotiation.cc:407] Connection header received
0806 16:06:30.659934 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:30.659937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:30.659999 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:30.660344 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:30.661380 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:30.661976 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:30.663019 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:30.663265 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:30.663396 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:30.666965 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:30.666969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:30.666971 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:30.667016 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:30.668956 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:30.669632 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:30.669637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:30.669638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:30.669747 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:30.670089 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:30.670093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:30.670095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:30.670441 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:30.670626 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:30.670960 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 16:06:30.671125 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
W20260806 16:06:30.672045 12537 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41272
I20260806 16:06:30.673508 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 12616
I20260806 16:06:30.691787 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 12789
I20260806 16:06:30.709537 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 12960
I20260806 16:06:30.727514 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 12494
2026-08-06T16:06:30Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2942 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:30 dist-test-slave-7z32 krb5kdc[13145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:30 dist-test-slave-7z32 krb5kdc[13145](info): set up 2 sockets
Aug 06 16:06:30 dist-test-slave-7z32 krb5kdc[13145](info): commencing operation
krb5kdc: starting...
W20260806 16:06:32.086167 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.323s user 0.001s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:32 dist-test-slave-7z32 krb5kdc[13145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:06:32.264179 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:41825
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:41825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:32.421207 13162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:32.421651 13162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:32.421739 13162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:32.426630 13162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:32.426776 13162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:32.426817 13162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:32.426872 13162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:32.426916 13162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:32.433735 13162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:41825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:41825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:32.435514 13162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:32.436970 13162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:32.448791 13171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:32.448864 13169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:32.448807 13168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:32.449118 13162 server_base.cc:1034] running on GCE node
I20260806 16:06:32.450066 13162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:32.451838 13162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:32.453073 13162 hybrid_clock.cc:648] HybridClock initialized: now 1786032392453031 us; error 60 us; skew 500 ppm
Aug 06 16:06:32 dist-test-slave-7z32 krb5kdc[13145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032392, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:32.457028 13162 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:06:32.458454 13162 webserver.cc:460] Webserver started at http://127.8.49.62:44489/ using document root <none> and password file <none>
I20260806 16:06:32.459338 13162 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:32.459453 13162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:32.459757 13162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:32.462656 13162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "93df8db847a84cfc8e5be584915224de"
format_stamp: "Formatted at 2026-08-06 16:06:32 on dist-test-slave-7z32"
server_key: "750b08e9e55306dcbdcd0eb2610ee9f9"
server_key_iv: "3ec3b035bde424c2fc9e1785a381d0fe"
server_key_version: "encryptionkey@0"
I20260806 16:06:32.463661 13162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "93df8db847a84cfc8e5be584915224de"
format_stamp: "Formatted at 2026-08-06 16:06:32 on dist-test-slave-7z32"
server_key: "750b08e9e55306dcbdcd0eb2610ee9f9"
server_key_iv: "3ec3b035bde424c2fc9e1785a381d0fe"
server_key_version: "encryptionkey@0"
I20260806 16:06:32.469218 13162 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:06:32.473336 13180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:32.475344 13162 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:32.475569 13162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "93df8db847a84cfc8e5be584915224de"
format_stamp: "Formatted at 2026-08-06 16:06:32 on dist-test-slave-7z32"
server_key: "750b08e9e55306dcbdcd0eb2610ee9f9"
server_key_iv: "3ec3b035bde424c2fc9e1785a381d0fe"
server_key_version: "encryptionkey@0"
I20260806 16:06:32.475746 13162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:32.494331 13162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:32.498799 13162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:32.499128 13162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:32.509263 13162 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:41825
I20260806 16:06:32.509297 13249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:41825 every 8 connection(s)
I20260806 16:06:32.510864 13162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:32.514495 13250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:32.516139 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 13162
I20260806 16:06:32.516283 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:32.516690 8388 external_mini_cluster.cc:1428] Setting key 5f2122c3cf792cf697e724984b24c3d3
I20260806 16:06:32.522464 13250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de: Bootstrap starting.
I20260806 16:06:32.525521 13250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:32.526463 13250 log.cc:824] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:32.529855 13250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de: No bootstrap required, opened a new log
Aug 06 16:06:32 dist-test-slave-7z32 krb5kdc[13145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:32.534107 13250 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df8db847a84cfc8e5be584915224de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41825 } }
I20260806 16:06:32.534663 13250 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:32.534873 13250 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93df8db847a84cfc8e5be584915224de, State: Initialized, Role: FOLLOWER
I20260806 16:06:32.535596 13250 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df8db847a84cfc8e5be584915224de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41825 } }
I20260806 16:06:32.535780 13250 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:32.535888 13250 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:32.536010 13250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:32.537531 13250 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df8db847a84cfc8e5be584915224de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41825 } }
I20260806 16:06:32.537999 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:32.519263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42180 (local address 127.8.49.62:41825)
0806 16:06:32.519953 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:32.519978 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:32.520049 (+ 71us) server_negotiation.cc:407] Connection header received
0806 16:06:32.521098 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:32.521126 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:32.521621 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:32.522060 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:32.523489 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:32.525028 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:32.526296 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:32.527077 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:32.527231 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:32.530907 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:32.530916 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:32.530935 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:32.530975 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:32.533667 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:32.534569 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:32.534575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:32.534582 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:32.534873 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:32.535355 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:32.535360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:32.535362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:32.535994 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:32.536256 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:32.536606 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 16:06:32.536976 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":214,"threads_started":1}
I20260806 16:06:32.538018 13250 leader_election.cc:302] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93df8db847a84cfc8e5be584915224de; no voters:
I20260806 16:06:32.538589 13250 leader_election.cc:288] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:32.538708 13256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:32.538985 13256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [term 1 LEADER]: Becoming Leader. State: Replica: 93df8db847a84cfc8e5be584915224de, State: Running, Role: LEADER
I20260806 16:06:32.539394 13256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df8db847a84cfc8e5be584915224de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41825 } }
I20260806 16:06:32.540410 13250 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:32.540973 13256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93df8db847a84cfc8e5be584915224de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df8db847a84cfc8e5be584915224de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41825 } } }
I20260806 16:06:32.541126 13256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:32.541404 13256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93df8db847a84cfc8e5be584915224de. Latest consensus state: current_term: 1 leader_uuid: "93df8db847a84cfc8e5be584915224de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df8db847a84cfc8e5be584915224de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41825 } } }
I20260806 16:06:32.541527 13256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:32.541765 13261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:32.545601 13261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:32.553304 13261 catalog_manager.cc:1347] Generated new cluster ID: f50de30a2d4146c6908ec27dd40f1764
I20260806 16:06:32.553483 13261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:32.571810 13261 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:32.573050 13261 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:32.585346 13261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93df8db847a84cfc8e5be584915224de: Generated new TSK 0
I20260806 16:06:32.586228 13261 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:06:32.635207 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41825
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:32.773556 13288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:32.773970 13288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:32.774062 13288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:32.778472 13288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:32.778594 13288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:32.778694 13288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:32.784689 13288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:32.785969 13288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:32.787173 13288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:32.799188 13293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:32.799427 13296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:32.799427 13294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:32.799973 13288 server_base.cc:1034] running on GCE node
I20260806 16:06:32.800609 13288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:32.801669 13288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:32.803231 13288 hybrid_clock.cc:648] HybridClock initialized: now 1786032392803190 us; error 63 us; skew 500 ppm
Aug 06 16:06:32 dist-test-slave-7z32 krb5kdc[13145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032392, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:32.808068 13288 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:06:32.809458 13288 webserver.cc:460] Webserver started at http://127.8.49.1:36881/ using document root <none> and password file <none>
I20260806 16:06:32.810267 13288 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:32.810380 13288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:32.810671 13288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:32.813935 13288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "ae557b44b12d48c4a04d4c7687816842"
format_stamp: "Formatted at 2026-08-06 16:06:32 on dist-test-slave-7z32"
server_key: "b0bcb4efd61091daeb0733f1ac48d75f"
server_key_iv: "ae2252d778f71c95e3e9f94c16ded7dc"
server_key_version: "encryptionkey@0"
I20260806 16:06:32.814735 13288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "ae557b44b12d48c4a04d4c7687816842"
format_stamp: "Formatted at 2026-08-06 16:06:32 on dist-test-slave-7z32"
server_key: "b0bcb4efd61091daeb0733f1ac48d75f"
server_key_iv: "ae2252d778f71c95e3e9f94c16ded7dc"
server_key_version: "encryptionkey@0"
I20260806 16:06:32.821410 13288 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:06:32.825392 13305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:32.827447 13288 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:32.827631 13288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "ae557b44b12d48c4a04d4c7687816842"
format_stamp: "Formatted at 2026-08-06 16:06:32 on dist-test-slave-7z32"
server_key: "b0bcb4efd61091daeb0733f1ac48d75f"
server_key_iv: "ae2252d778f71c95e3e9f94c16ded7dc"
server_key_version: "encryptionkey@0"
I20260806 16:06:32.827786 13288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:32.844420 13288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:32.849336 13288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:32.849570 13288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:32.850404 13288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:32.851755 13288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:32.851845 13288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:32.851940 13288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:32.851987 13288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:32.865362 13288 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:35303
I20260806 16:06:32.865386 13459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:35303 every 8 connection(s)
I20260806 16:06:32.866680 13288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:32.867491 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 13288
I20260806 16:06:32.867660 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:32.868022 8388 external_mini_cluster.cc:1428] Setting key 9a969ec5fc3abbf0c12d19db8662fd75
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
Aug 06 16:06:32 dist-test-slave-7z32 krb5kdc[13145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032392, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:32.900760 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:32.869409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:46203 (local address 127.8.49.62:41825)
0806 16:06:32.869623 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:32.869629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:32.870792 (+ 1163us) server_negotiation.cc:407] Connection header received
0806 16:06:32.872462 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:32.872469 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:32.872562 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:32.872762 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:32.874373 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:32.875015 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:32.876931 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:32.877256 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:32.877405 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:32.895705 (+ 18300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:32.895710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:32.895713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:32.895757 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:32.897466 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:32.898265 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:32.898275 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:32.898277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:32.898414 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:32.898952 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:32.898956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:32.898957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:32.899322 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:32.899452 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:32.900298 (+ 846us) server_negotiation.cc:299] Negotiation successful
0806 16:06:32.900486 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 16:06:32.902190 13462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:32.869840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:41825 (local address 127.8.49.1:46203)
0806 16:06:32.870590 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:32.870650 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:32.872183 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:32.873025 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:32.873034 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:32.873577 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:32.874144 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:32.874158 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:32.875162 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:32.875168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:32.876714 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:32.876730 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:32.877529 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:32.877538 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:32.877825 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:32.892423 (+ 14598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:32.892465 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:32.895348 (+ 2883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:32.897708 (+ 2360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:32.897717 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:32.897741 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:32.898117 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:32.898573 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:32.898584 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:32.898586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:32.898783 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:32.899455 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:32.899467 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:32.899852 (+ 385us) client_negotiation.cc:769] Sending connection context
0806 16:06:32.900240 (+ 388us) client_negotiation.cc:240] Negotiation successful
0806 16:06:32.900822 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":220,"threads_started":1}
I20260806 16:06:32.904209 13460 heartbeater.cc:347] Connected to a master server at 127.8.49.62:41825
I20260806 16:06:32.904548 13460 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:32.905189 13460 heartbeater.cc:511] Master 127.8.49.62:41825 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:06:32.907720 13202 ts_manager.cc:194] Registered new tserver with Master: ae557b44b12d48c4a04d4c7687816842 (127.8.49.1:35303)
I20260806 16:06:32.907960 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41825
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:06:32.909813 13202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:46203
W20260806 16:06:33.060949 13465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:33.061272 13465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:33.061318 13465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:33.065608 13465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:33.065697 13465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:33.065761 13465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:33.071468 13465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:33.072661 13465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:33.073719 13465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:33.085070 13470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:33.085177 13474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:33.085615 13471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:33.085656 13465 server_base.cc:1034] running on GCE node
I20260806 16:06:33.086334 13465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:33.087332 13465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:33.088587 13465 hybrid_clock.cc:648] HybridClock initialized: now 1786032393088538 us; error 59 us; skew 500 ppm
Aug 06 16:06:33 dist-test-slave-7z32 krb5kdc[13145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032393, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:33.092724 13465 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:06:33.094091 13465 webserver.cc:460] Webserver started at http://127.8.49.2:41801/ using document root <none> and password file <none>
I20260806 16:06:33.094955 13465 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:33.095077 13465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:33.095387 13465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:33.098142 13465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "dd46f7e31685406fa74132c14b0d4323"
format_stamp: "Formatted at 2026-08-06 16:06:33 on dist-test-slave-7z32"
server_key: "06b042485bf3b15b2473eafef8bb69fb"
server_key_iv: "1d969de31c6f18b0d4f10e20d5d0c517"
server_key_version: "encryptionkey@0"
I20260806 16:06:33.098970 13465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "dd46f7e31685406fa74132c14b0d4323"
format_stamp: "Formatted at 2026-08-06 16:06:33 on dist-test-slave-7z32"
server_key: "06b042485bf3b15b2473eafef8bb69fb"
server_key_iv: "1d969de31c6f18b0d4f10e20d5d0c517"
server_key_version: "encryptionkey@0"
I20260806 16:06:33.105047 13465 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 16:06:33.109256 13484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:33.111449 13465 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:33.111682 13465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "dd46f7e31685406fa74132c14b0d4323"
format_stamp: "Formatted at 2026-08-06 16:06:33 on dist-test-slave-7z32"
server_key: "06b042485bf3b15b2473eafef8bb69fb"
server_key_iv: "1d969de31c6f18b0d4f10e20d5d0c517"
server_key_version: "encryptionkey@0"
I20260806 16:06:33.111953 13465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:33.141757 13465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:33.146344 13465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:33.146590 13465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:33.147539 13465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:33.148725 13465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:33.148787 13465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:33.148837 13465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:33.148855 13465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:33.162868 13465 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:35757
I20260806 16:06:33.162958 13648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:35757 every 8 connection(s)
I20260806 16:06:33.164252 13465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:33.170985 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 13465
I20260806 16:06:33.171183 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:33.171558 8388 external_mini_cluster.cc:1428] Setting key 2c9a686271d99b710e59c0d4d29143d1
Aug 06 16:06:33 dist-test-slave-7z32 krb5kdc[13145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032393, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:33.184643 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:33.166876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:56371 (local address 127.8.49.62:41825)
0806 16:06:33.167083 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:33.167087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:33.168280 (+ 1193us) server_negotiation.cc:407] Connection header received
0806 16:06:33.169956 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:33.169961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:33.170030 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:33.170125 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:33.171747 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.172365 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.174288 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.174678 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.174851 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:33.178701 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:33.178704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:33.178707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:33.178743 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:33.180814 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:33.182130 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:33.182134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:33.182135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:33.182250 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:33.182935 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:33.182938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:33.182939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:33.183196 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:33.183295 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:33.184322 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0806 16:06:33.184486 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 16:06:33.186528 13651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:33.167396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:41825 (local address 127.8.49.2:56371)
0806 16:06:33.168065 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:33.168125 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:33.169683 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:33.170356 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:33.170363 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:33.170809 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:33.171548 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:33.171564 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.172597 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.172603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:33.174093 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:33.174113 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.175056 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.175061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:33.175328 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:33.176199 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:33.176221 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:33.178385 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:33.180991 (+ 2606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:33.181002 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:33.181033 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:33.181905 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:33.182387 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:33.182394 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:33.182400 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:33.182697 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:33.183347 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:33.183361 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:33.183930 (+ 569us) client_negotiation.cc:769] Sending connection context
0806 16:06:33.184248 (+ 318us) client_negotiation.cc:240] Negotiation successful
0806 16:06:33.184838 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":242,"threads_started":1}
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:06:33.188730 13649 heartbeater.cc:347] Connected to a master server at 127.8.49.62:41825
I20260806 16:06:33.189148 13649 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:33.189915 13649 heartbeater.cc:511] Master 127.8.49.62:41825 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:06:33.191658 13202 ts_manager.cc:194] Registered new tserver with Master: dd46f7e31685406fa74132c14b0d4323 (127.8.49.2:35757)
I20260806 16:06:33.192812 13202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:56371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:06:33.216300 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41825
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:34851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:33.377331 13657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:33.377887 13657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:33.377996 13657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:33.383399 13657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:33.383589 13657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:33.383719 13657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:33.390718 13657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:34851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:33.392438 13657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:33.393891 13657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:33.410342 13667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:33.410337 13663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:33.410383 13664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:33.411051 13657 server_base.cc:1034] running on GCE node
I20260806 16:06:33.411777 13657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:33.412678 13657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:33.416026 13657 hybrid_clock.cc:648] HybridClock initialized: now 1786032393415951 us; error 104 us; skew 500 ppm
Aug 06 16:06:33 dist-test-slave-7z32 krb5kdc[13145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032393, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:33.420303 13657 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:06:33.421626 13657 webserver.cc:460] Webserver started at http://127.8.49.3:39703/ using document root <none> and password file <none>
I20260806 16:06:33.422375 13657 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:33.422495 13657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:33.422791 13657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:33.425390 13657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "dde81002e52c4b638615c097007c71c7"
format_stamp: "Formatted at 2026-08-06 16:06:33 on dist-test-slave-7z32"
server_key: "dcdf31c5002c22247d9ffb292687939f"
server_key_iv: "c00e0df0a2479acd236789750d4bd7a4"
server_key_version: "encryptionkey@0"
I20260806 16:06:33.426101 13657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "dde81002e52c4b638615c097007c71c7"
format_stamp: "Formatted at 2026-08-06 16:06:33 on dist-test-slave-7z32"
server_key: "dcdf31c5002c22247d9ffb292687939f"
server_key_iv: "c00e0df0a2479acd236789750d4bd7a4"
server_key_version: "encryptionkey@0"
I20260806 16:06:33.431288 13657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 16:06:33.435111 13679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:33.436827 13657 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:33.437024 13657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "dde81002e52c4b638615c097007c71c7"
format_stamp: "Formatted at 2026-08-06 16:06:33 on dist-test-slave-7z32"
server_key: "dcdf31c5002c22247d9ffb292687939f"
server_key_iv: "c00e0df0a2479acd236789750d4bd7a4"
server_key_version: "encryptionkey@0"
I20260806 16:06:33.437172 13657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:33.459342 13657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:33.463474 13657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:33.463732 13657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:33.464462 13657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:33.465519 13657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:33.465605 13657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:33.465692 13657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:33.465754 13657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:33.477514 13657 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:34475
I20260806 16:06:33.477557 13844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:34475 every 8 connection(s)
I20260806 16:06:33.478740 13657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:33.489054 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 13657
I20260806 16:06:33.489212 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:33.489583 8388 external_mini_cluster.cc:1428] Setting key f6f51bef2a06080e57b5d1030cadb9b5
Aug 06 16:06:33 dist-test-slave-7z32 krb5kdc[13145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032393, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:33.497890 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:33.481480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:56351 (local address 127.8.49.62:41825)
0806 16:06:33.481687 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:33.481693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:33.482970 (+ 1277us) server_negotiation.cc:407] Connection header received
0806 16:06:33.484722 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:33.484725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:33.484788 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:33.484893 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:33.486701 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.487380 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.488946 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.489263 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.489428 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:33.493082 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:33.493085 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:33.493088 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:33.493115 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:33.494979 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:33.495920 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:33.495923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:33.495925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:33.496039 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:33.496521 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:33.496523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:33.496525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:33.496742 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:33.496837 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:33.497613 (+ 776us) server_negotiation.cc:299] Negotiation successful
0806 16:06:33.497743 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 16:06:33.499315 13849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:33.482040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:41825 (local address 127.8.49.3:56351)
0806 16:06:33.482690 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:33.482764 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:33.484462 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:33.485124 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:33.485133 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:33.485830 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:33.486517 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:33.486535 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.487578 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.487582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:33.488730 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:33.488748 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.489520 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.489529 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:33.489847 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:33.490647 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:33.490669 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:33.492872 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:33.495216 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:33.495225 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:33.495260 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:33.495780 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:33.496184 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:33.496188 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:33.496190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:33.496395 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:33.496887 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:33.496895 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:33.497282 (+ 387us) client_negotiation.cc:769] Sending connection context
0806 16:06:33.497522 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 16:06:33.497978 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":136,"threads_started":1}
I20260806 16:06:33.500959 13847 heartbeater.cc:347] Connected to a master server at 127.8.49.62:41825
I20260806 16:06:33.501370 13847 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:33.502208 13847 heartbeater.cc:511] Master 127.8.49.62:41825 requested a full tablet report, sending...
I20260806 16:06:33.504009 13201 ts_manager.cc:194] Registered new tserver with Master: dde81002e52c4b638615c097007c71c7 (127.8.49.3:34475)
I20260806 16:06:33.505000 13201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:56351
I20260806 16:06:33.505391 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 16:06:33 dist-test-slave-7z32 krb5kdc[13145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 16:06:33 dist-test-slave-7z32 krb5kdc[13145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:06:33.544924 13854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:33.531558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52104 (local address 127.8.49.1:35303)
0806 16:06:33.532097 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:33.532109 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:33.532149 (+ 40us) server_negotiation.cc:407] Connection header received
0806 16:06:33.532235 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:33.532242 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:33.532518 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:33.532691 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:33.533642 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.534573 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.535717 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.536211 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.536338 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:33.539422 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:33.539435 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:33.539457 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:33.539498 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:33.542224 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:33.543044 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:33.543052 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:33.543053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:33.543182 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:33.543510 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:33.543512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:33.543513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:33.543909 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:33.544125 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:33.544399 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 16:06:33.544663 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":138,"threads_started":1}
Aug 06 16:06:33 dist-test-slave-7z32 krb5kdc[13145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:33.559834 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:33.549005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42194 (local address 127.8.49.62:41825)
0806 16:06:33.549175 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:33.549179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:33.549220 (+ 41us) server_negotiation.cc:407] Connection header received
0806 16:06:33.549382 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:33.549385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:33.549436 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:33.549522 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:33.550252 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.550967 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.551875 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:33.552125 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:33.552216 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:33.555414 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:33.555418 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:33.555421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:33.555458 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:33.557269 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:33.557892 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:33.557902 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:33.557904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:33.558000 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:33.558572 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:33.558577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:33.558579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:33.559019 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:33.559222 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:33.559468 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 16:06:33.559619 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 16:06:33.560482 13201 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42194
I20260806 16:06:33.561597 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 13288
I20260806 16:06:33.580921 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 13465
I20260806 16:06:33.591934 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 13657
I20260806 16:06:33.613471 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 13162
2026-08-06T16:06:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2887 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:33 dist-test-slave-7z32 krb5kdc[13859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:33 dist-test-slave-7z32 krb5kdc[13859](info): set up 2 sockets
Aug 06 16:06:33 dist-test-slave-7z32 krb5kdc[13859](info): commencing operation
krb5kdc: starting...
W20260806 16:06:34.957798 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:35 dist-test-slave-7z32 krb5kdc[13859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:06:35.140173 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:46219
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:46219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:35.297776 13875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:35.298269 13875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:35.298388 13875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:35.303539 13875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:35.303731 13875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:35.303779 13875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:35.303828 13875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:35.303897 13875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:35.311410 13875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:46219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:46219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:35.313261 13875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:35.314899 13875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:35.326508 13886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:35.326508 13883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:35.326604 13875 server_base.cc:1034] running on GCE node
W20260806 16:06:35.326633 13882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:35.327761 13875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:35.329407 13875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:35.330654 13875 hybrid_clock.cc:648] HybridClock initialized: now 1786032395330629 us; error 49 us; skew 500 ppm
Aug 06 16:06:35 dist-test-slave-7z32 krb5kdc[13859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032395, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:35.334694 13875 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:06:35.335902 13875 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:35.336019 13875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:35.336328 13875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:35.339244 13875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "7bcc606c6c69462392fd58eb2b843cda"
format_stamp: "Formatted at 2026-08-06 16:06:35 on dist-test-slave-7z32"
server_key: "be004134c8ac63fbf19dab6e7ffb558a"
server_key_iv: "3c24ebb924fd768532ddd7d9dcb7ec84"
server_key_version: "encryptionkey@0"
I20260806 16:06:35.339972 13875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "7bcc606c6c69462392fd58eb2b843cda"
format_stamp: "Formatted at 2026-08-06 16:06:35 on dist-test-slave-7z32"
server_key: "be004134c8ac63fbf19dab6e7ffb558a"
server_key_iv: "3c24ebb924fd768532ddd7d9dcb7ec84"
server_key_version: "encryptionkey@0"
I20260806 16:06:35.345242 13875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 16:06:35.349246 13894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:35.351622 13875 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:35.351871 13875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "7bcc606c6c69462392fd58eb2b843cda"
format_stamp: "Formatted at 2026-08-06 16:06:35 on dist-test-slave-7z32"
server_key: "be004134c8ac63fbf19dab6e7ffb558a"
server_key_iv: "3c24ebb924fd768532ddd7d9dcb7ec84"
server_key_version: "encryptionkey@0"
I20260806 16:06:35.352061 13875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:35.428314 13875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:35.432660 13875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:35.432933 13875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:35.443118 13875 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:46219
I20260806 16:06:35.443593 13969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:46219 every 8 connection(s)
I20260806 16:06:35.444180 13875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:35.447916 13972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:35.453524 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 13875
I20260806 16:06:35.453692 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:35.454042 8388 external_mini_cluster.cc:1428] Setting key 942a6b1ee28649d1dbb7814455d17fa0
I20260806 16:06:35.454584 13972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda: Bootstrap starting.
I20260806 16:06:35.457638 13972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:35.458515 13972 log.cc:824] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:35.460737 13972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda: No bootstrap required, opened a new log
I20260806 16:06:35.464574 13972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc606c6c69462392fd58eb2b843cda" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46219 } }
I20260806 16:06:35.464838 13972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:35.464913 13972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bcc606c6c69462392fd58eb2b843cda, State: Initialized, Role: FOLLOWER
I20260806 16:06:35.465432 13972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc606c6c69462392fd58eb2b843cda" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46219 } }
I20260806 16:06:35.465581 13972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:35.465677 13972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:35.465790 13972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [term 0 FOLLOWER]: Advancing to term 1
Aug 06 16:06:35 dist-test-slave-7z32 krb5kdc[13859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:35.466900 13972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc606c6c69462392fd58eb2b843cda" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46219 } }
I20260806 16:06:35.467307 13972 leader_election.cc:302] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bcc606c6c69462392fd58eb2b843cda; no voters:
I20260806 16:06:35.467711 13972 leader_election.cc:288] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:35.467904 13979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:35.468299 13979 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [term 1 LEADER]: Becoming Leader. State: Replica: 7bcc606c6c69462392fd58eb2b843cda, State: Running, Role: LEADER
I20260806 16:06:35.468731 13979 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc606c6c69462392fd58eb2b843cda" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46219 } }
I20260806 16:06:35.469347 13972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:35.471436 13981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bcc606c6c69462392fd58eb2b843cda. Latest consensus state: current_term: 1 leader_uuid: "7bcc606c6c69462392fd58eb2b843cda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc606c6c69462392fd58eb2b843cda" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46219 } } }
I20260806 16:06:35.471588 13981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:35.472762 13980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bcc606c6c69462392fd58eb2b843cda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc606c6c69462392fd58eb2b843cda" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46219 } } }
I20260806 16:06:35.472887 13980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:35.472980 13994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:35.474714 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:35.456274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49462 (local address 127.8.49.62:46219)
0806 16:06:35.456924 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:35.456949 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:35.457002 (+ 53us) server_negotiation.cc:407] Connection header received
0806 16:06:35.458234 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:35.458268 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:35.458910 (+ 642us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:35.459425 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:35.460825 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:35.461986 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:35.463063 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:35.463645 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:35.463847 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:35.466994 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:35.467000 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:35.467023 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:35.467089 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:35.469689 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:35.470388 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:35.470397 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:35.470406 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:35.470608 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:35.471045 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:35.471049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:35.471051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:35.471801 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:35.472855 (+ 1054us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:35.473085 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 16:06:35.473491 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":211,"threads_started":1}
I20260806 16:06:35.476846 13994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:35.485735 13994 catalog_manager.cc:1347] Generated new cluster ID: 4b025e8097a6407aa4244056cc25fd11
I20260806 16:06:35.485850 13994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:35.499622 13994 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:35.501020 13994 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:35.511492 13994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda: Generated new TSK 0
I20260806 16:06:35.512485 13994 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:06:35.560426 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46219
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:35.730249 14009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:35.730764 14009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:35.730902 14009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:35.735963 14009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:35.736106 14009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:35.736236 14009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:35.743055 14009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:35.744737 14009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:35.746194 14009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:06:35.760208 14009 server_base.cc:1034] running on GCE node
W20260806 16:06:35.760046 14015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:35.760026 14019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:35.760020 14022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:35.761211 14009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:35.762310 14009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:35.763725 14009 hybrid_clock.cc:648] HybridClock initialized: now 1786032395763668 us; error 72 us; skew 500 ppm
Aug 06 16:06:35 dist-test-slave-7z32 krb5kdc[13859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032395, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:35.768515 14009 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:06:35.769670 14009 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:35.769793 14009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:35.770098 14009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:35.773449 14009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "c8c32242398f4de08c0401d0e4122eb7"
format_stamp: "Formatted at 2026-08-06 16:06:35 on dist-test-slave-7z32"
server_key: "6b69b0e8f3e23215bb76456a93e714db"
server_key_iv: "f1486740de314a698daddaea16fe9ac1"
server_key_version: "encryptionkey@0"
I20260806 16:06:35.774291 14009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "c8c32242398f4de08c0401d0e4122eb7"
format_stamp: "Formatted at 2026-08-06 16:06:35 on dist-test-slave-7z32"
server_key: "6b69b0e8f3e23215bb76456a93e714db"
server_key_iv: "f1486740de314a698daddaea16fe9ac1"
server_key_version: "encryptionkey@0"
I20260806 16:06:35.779726 14009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:06:35.783697 14029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:35.785902 14009 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:35.786070 14009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "c8c32242398f4de08c0401d0e4122eb7"
format_stamp: "Formatted at 2026-08-06 16:06:35 on dist-test-slave-7z32"
server_key: "6b69b0e8f3e23215bb76456a93e714db"
server_key_iv: "f1486740de314a698daddaea16fe9ac1"
server_key_version: "encryptionkey@0"
I20260806 16:06:35.786211 14009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:35.812799 14009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:35.817819 14009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:35.818041 14009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:35.818727 14009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:35.819891 14009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:35.819952 14009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:35.820000 14009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:35.820017 14009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:35.832594 14009 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:34661
I20260806 16:06:35.832823 14190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:34661 every 8 connection(s)
I20260806 16:06:35.833453 14009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:35.833877 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 14009
I20260806 16:06:35.834028 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:35.834383 8388 external_mini_cluster.cc:1428] Setting key 41439ac2d9c8183f915c6f40b9cd3ef1
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
Aug 06 16:06:35 dist-test-slave-7z32 krb5kdc[13859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032395, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:35.860486 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:35.836369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:35187 (local address 127.8.49.62:46219)
0806 16:06:35.836550 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:35.836555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:35.838197 (+ 1642us) server_negotiation.cc:407] Connection header received
0806 16:06:35.841012 (+ 2815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:35.841019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:35.841134 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:35.841284 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:35.843411 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:35.844541 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:35.846315 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:35.846870 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:35.847054 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:35.854942 (+ 7888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:35.854951 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:35.854957 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:35.855018 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:35.857095 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:35.858315 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:35.858320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:35.858323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:35.858466 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:35.858958 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:35.858962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:35.858964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:35.859239 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:35.859364 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:35.860071 (+ 707us) server_negotiation.cc:299] Negotiation successful
0806 16:06:35.860275 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:06:35.861690 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:35.837014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46219 (local address 127.8.49.1:35187)
0806 16:06:35.837903 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:35.837978 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:35.840506 (+ 2528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:35.841667 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:35.841678 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:35.842269 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:35.843130 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:35.843153 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:35.844907 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:35.844911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:35.846096 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:35.846114 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:35.847156 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:35.847160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:35.847445 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:35.848369 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:35.848394 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:35.854494 (+ 6100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:35.857332 (+ 2838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:35.857347 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:35.857383 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:35.858108 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:35.858586 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:35.858590 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:35.858592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:35.858792 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:35.859380 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:35.859391 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:35.859767 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 16:06:35.860011 (+ 244us) client_negotiation.cc:240] Negotiation successful
0806 16:06:35.860499 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":224,"threads_started":1}
I20260806 16:06:35.863813 14191 heartbeater.cc:347] Connected to a master server at 127.8.49.62:46219
I20260806 16:06:35.864187 14191 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:35.864878 14191 heartbeater.cc:511] Master 127.8.49.62:46219 requested a full tablet report, sending...
I20260806 16:06:35.867301 13917 ts_manager.cc:194] Registered new tserver with Master: c8c32242398f4de08c0401d0e4122eb7 (127.8.49.1:34661)
I20260806 16:06:35.868929 13917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:35187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:06:35.880815 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46219
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:36.038203 14200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:36.038607 14200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:36.038666 14200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:36.044203 14200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:36.044361 14200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:36.044500 14200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:36.050634 14200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:36.052031 14200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:36.053444 14200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:36.065814 14206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:36.065804 14207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:36.065805 14211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:36.066411 14200 server_base.cc:1034] running on GCE node
I20260806 16:06:36.067148 14200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:36.068114 14200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:36.069458 14200 hybrid_clock.cc:648] HybridClock initialized: now 1786032396069408 us; error 76 us; skew 500 ppm
Aug 06 16:06:36 dist-test-slave-7z32 krb5kdc[13859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032396, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:36.073935 14200 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:06:36.075014 14200 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:36.075160 14200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:36.075424 14200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:36.078132 14200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "3a7486e882f14306bdf25aa9a197ce19"
format_stamp: "Formatted at 2026-08-06 16:06:36 on dist-test-slave-7z32"
server_key: "bb897bbe98cdc3575b2004d75fa8119e"
server_key_iv: "ab5ad37666e370c0ba9cf63387280385"
server_key_version: "encryptionkey@0"
I20260806 16:06:36.078933 14200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "3a7486e882f14306bdf25aa9a197ce19"
format_stamp: "Formatted at 2026-08-06 16:06:36 on dist-test-slave-7z32"
server_key: "bb897bbe98cdc3575b2004d75fa8119e"
server_key_iv: "ab5ad37666e370c0ba9cf63387280385"
server_key_version: "encryptionkey@0"
I20260806 16:06:36.084052 14200 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:06:36.087728 14221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:36.089834 14200 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:36.090091 14200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "3a7486e882f14306bdf25aa9a197ce19"
format_stamp: "Formatted at 2026-08-06 16:06:36 on dist-test-slave-7z32"
server_key: "bb897bbe98cdc3575b2004d75fa8119e"
server_key_iv: "ab5ad37666e370c0ba9cf63387280385"
server_key_version: "encryptionkey@0"
I20260806 16:06:36.090258 14200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:36.103588 14200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:36.107738 14200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:36.108009 14200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:36.108773 14200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:36.109875 14200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:36.109964 14200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:36.110045 14200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:36.110095 14200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:36.122527 14200 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:33603
I20260806 16:06:36.122557 14382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:33603 every 8 connection(s)
I20260806 16:06:36.123330 14200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:36.132838 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 14200
I20260806 16:06:36.132987 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:36.133289 8388 external_mini_cluster.cc:1428] Setting key 91a35194b2e7e97d710a2efd75823bb4
Aug 06 16:06:36 dist-test-slave-7z32 krb5kdc[13859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032396, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:36.144028 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.125869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:56373 (local address 127.8.49.62:46219)
0806 16:06:36.126617 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:36.126624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:36.127723 (+ 1099us) server_negotiation.cc:407] Connection header received
0806 16:06:36.129858 (+ 2135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:36.129862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:36.129940 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:36.130057 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:36.131789 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.132416 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.134258 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.134521 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.134699 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.138482 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:36.138486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:36.138541 (+ 55us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:36.138576 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:36.140652 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:36.141653 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:36.141656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:36.141658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:36.141769 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:36.142321 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:36.142324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:36.142326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:36.142569 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:36.142664 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:36.143687 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0806 16:06:36.143851 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601}
I20260806 16:06:36.145763 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.126576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46219 (local address 127.8.49.2:56373)
0806 16:06:36.127492 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:36.127580 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:36.129518 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:36.130331 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:36.130341 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:36.130900 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:36.131565 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.131582 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.132614 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.132618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.134034 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.134048 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.134777 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.134782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.135044 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.135803 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:36.135829 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:36.138159 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:36.140861 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:36.140874 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:36.140903 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:36.141454 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:36.141900 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:36.141905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:36.141907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:36.142182 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:36.142710 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:36.142723 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:36.143282 (+ 559us) client_negotiation.cc:769] Sending connection context
0806 16:06:36.143614 (+ 332us) client_negotiation.cc:240] Negotiation successful
0806 16:06:36.144197 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":244,"threads_started":1}
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:06:36.147662 14383 heartbeater.cc:347] Connected to a master server at 127.8.49.62:46219
I20260806 16:06:36.148064 14383 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:36.148748 14383 heartbeater.cc:511] Master 127.8.49.62:46219 requested a full tablet report, sending...
I20260806 16:06:36.150338 13917 ts_manager.cc:194] Registered new tserver with Master: 3a7486e882f14306bdf25aa9a197ce19 (127.8.49.2:33603)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:06:36.151193 13917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:56373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:06:36.176388 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46219
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:36.317533 14389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:36.317940 14389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:36.318032 14389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:36.323287 14389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:36.323441 14389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:36.323565 14389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:36.329931 14389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:36.331490 14389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:36.332823 14389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:36.345161 14400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:36.345132 14398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:36.345132 14395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:36.345395 14389 server_base.cc:1034] running on GCE node
I20260806 16:06:36.346365 14389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:36.347426 14389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:36.348682 14389 hybrid_clock.cc:648] HybridClock initialized: now 1786032396348659 us; error 34 us; skew 500 ppm
Aug 06 16:06:36 dist-test-slave-7z32 krb5kdc[13859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032396, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:36.353030 14389 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:06:36.354787 14389 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:36.354985 14389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:36.355288 14389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:36.359122 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "17327227f08848b99763a79987bacaf6"
format_stamp: "Formatted at 2026-08-06 16:06:36 on dist-test-slave-7z32"
server_key: "3da2b2c2d1f1e4890e1a9bd55f79d592"
server_key_iv: "4144006c4c5f79b8b4d29d172a234cf2"
server_key_version: "encryptionkey@0"
I20260806 16:06:36.359982 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "17327227f08848b99763a79987bacaf6"
format_stamp: "Formatted at 2026-08-06 16:06:36 on dist-test-slave-7z32"
server_key: "3da2b2c2d1f1e4890e1a9bd55f79d592"
server_key_iv: "4144006c4c5f79b8b4d29d172a234cf2"
server_key_version: "encryptionkey@0"
I20260806 16:06:36.365811 14389 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:06:36.370077 14409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:36.372727 14389 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260806 16:06:36.372916 14389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "17327227f08848b99763a79987bacaf6"
format_stamp: "Formatted at 2026-08-06 16:06:36 on dist-test-slave-7z32"
server_key: "3da2b2c2d1f1e4890e1a9bd55f79d592"
server_key_iv: "4144006c4c5f79b8b4d29d172a234cf2"
server_key_version: "encryptionkey@0"
I20260806 16:06:36.373070 14389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:36.405115 14389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:36.411131 14389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:36.411393 14389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:36.412174 14389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:36.413344 14389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:36.413415 14389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:36.413473 14389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:36.413508 14389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:36.425907 14389 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:43281
I20260806 16:06:36.425940 14569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:43281 every 8 connection(s)
I20260806 16:06:36.426700 14389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:36.430375 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 14389
I20260806 16:06:36.430560 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:36.430938 8388 external_mini_cluster.cc:1428] Setting key 178898e8fbdbcea32430b1ff7553ffb8
Aug 06 16:06:36 dist-test-slave-7z32 krb5kdc[13859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032396, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:36.448683 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.429774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:60545 (local address 127.8.49.62:46219)
0806 16:06:36.430024 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:36.430028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:36.431502 (+ 1474us) server_negotiation.cc:407] Connection header received
0806 16:06:36.433990 (+ 2488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:36.434044 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:36.434119 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:36.434235 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:36.436664 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.437285 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.439016 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.439257 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.439361 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.443104 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:36.443109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:36.443112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:36.443157 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:36.445487 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:36.446540 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:36.446546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:36.446548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:36.446707 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:36.447288 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:36.447291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:36.447292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:36.447537 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:36.447630 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:36.448385 (+ 755us) server_negotiation.cc:299] Negotiation successful
0806 16:06:36.448532 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 16:06:36.449877 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.430154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46219 (local address 127.8.49.3:60545)
0806 16:06:36.431178 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:36.431250 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:36.433630 (+ 2380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:36.434551 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:36.434562 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:36.435403 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:36.436422 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.436447 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.437487 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.437491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.438766 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.438779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.439463 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.439466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.439727 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.440464 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:36.440486 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:36.442557 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:36.445711 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:36.445721 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:36.445741 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:36.446253 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:36.446865 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:36.446869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:36.446871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:36.447113 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:36.447657 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:36.447664 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:36.448053 (+ 389us) client_negotiation.cc:769] Sending connection context
0806 16:06:36.448281 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 16:06:36.448705 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":210,"threads_started":1}
I20260806 16:06:36.451340 14570 heartbeater.cc:347] Connected to a master server at 127.8.49.62:46219
I20260806 16:06:36.451845 14570 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:36.452607 14570 heartbeater.cc:511] Master 127.8.49.62:46219 requested a full tablet report, sending...
I20260806 16:06:36.454314 13917 ts_manager.cc:194] Registered new tserver with Master: 17327227f08848b99763a79987bacaf6 (127.8.49.3:43281)
I20260806 16:06:36.455156 13917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:60545
I20260806 16:06:36.459262 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:06:36.472404 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.462034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49466 (local address 127.8.49.62:46219)
0806 16:06:36.462286 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:36.462292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:36.462477 (+ 185us) server_negotiation.cc:407] Connection header received
0806 16:06:36.462675 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:36.462679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:36.462754 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:36.462880 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:36.463811 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.464408 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.465327 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.465696 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.465948 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.467823 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:36.467828 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:36.467831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:36.467872 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:36.469942 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:36.470513 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:36.470517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:36.470519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:36.470618 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:36.471277 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:36.471281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:36.471282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:36.471587 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:36.471715 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:36.472095 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 16:06:36.472232 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 16:06:36.476451 13917 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:36.480559 13917 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:36.501009 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.493171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:43281 (local address 127.0.0.1:51838)
0806 16:06:36.494285 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:36.494347 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:36.494522 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:36.495633 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:36.495640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:36.495664 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:36.495845 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.495855 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.497438 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.497443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.499428 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.499442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.500348 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.500351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.500464 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.500510 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 16:06:36.500638 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 16:06:36.500798 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":242,"thread_start_us":214,"threads_started":1}
I20260806 16:06:36.501384 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.493405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34906 (local address 127.8.49.2:33603)
0806 16:06:36.494199 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:36.494219 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:36.494978 (+ 759us) server_negotiation.cc:407] Connection header received
0806 16:06:36.495132 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:36.495141 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:36.495566 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:36.495820 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:36.496177 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.497192 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.499590 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.500408 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.500518 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.500646 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:36.501056 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 16:06:36.501191 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":206,"threads_started":1}
I20260806 16:06:36.501632 14591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.493521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.8.49.3:43281)
0806 16:06:36.495047 (+ 1526us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:36.495066 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:36.495100 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:06:36.495191 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:36.495197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:36.495467 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:36.495645 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:36.496088 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.497269 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.499607 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.500203 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.500341 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.500534 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:36.501250 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 16:06:36.501370 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1375,"thread_start_us":131,"threads_started":1}
I20260806 16:06:36.501729 14595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.494598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36144 (local address 127.8.49.1:34661)
0806 16:06:36.495342 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:36.495349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:36.495381 (+ 32us) server_negotiation.cc:407] Connection header received
0806 16:06:36.495470 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:36.495475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:36.495743 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:36.495920 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:36.496472 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.497678 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.499276 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.500209 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.500359 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.500604 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:36.501399 (+ 795us) server_negotiation.cc:299] Negotiation successful
0806 16:06:36.501549 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":143,"threads_started":1}
I20260806 16:06:36.501258 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.494197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:34661 (local address 127.0.0.1:36144)
0806 16:06:36.494795 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:36.494812 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:36.495017 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:36.496087 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:36.496090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:36.496138 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:36.496267 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.496274 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.497798 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.497805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.499088 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.499100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.500694 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.500698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.500832 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.500878 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 16:06:36.500958 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 16:06:36.501074 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":101,"thread_start_us":150,"threads_started":1}
I20260806 16:06:36.501919 14592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.493228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:33603 (local address 127.0.0.1:34906)
0806 16:06:36.494630 (+ 1402us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:36.494650 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:36.494815 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:36.495875 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:36.495881 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:36.495904 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:36.496027 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.496035 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.497375 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.497378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.499428 (+ 2050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.499442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.500703 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.500707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.500856 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.500893 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 16:06:36.500957 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 16:06:36.501778 (+ 821us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1323,"mutex_wait_us":1098,"thread_start_us":116,"threads_started":1}
I20260806 16:06:36.505800 14094 tablet_service.cc:1467] Processing CreateTablet for tablet 88ad5c98350d4c44949e5947a0312af0 (DEFAULT_TABLE table=test-table [id=5e6cbe0169474e3aac421d59125936f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:36.505808 14283 tablet_service.cc:1467] Processing CreateTablet for tablet 88ad5c98350d4c44949e5947a0312af0 (DEFAULT_TABLE table=test-table [id=5e6cbe0169474e3aac421d59125936f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:36.507189 14094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88ad5c98350d4c44949e5947a0312af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:36.507477 14283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88ad5c98350d4c44949e5947a0312af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:36.507490 14478 tablet_service.cc:1467] Processing CreateTablet for tablet 88ad5c98350d4c44949e5947a0312af0 (DEFAULT_TABLE table=test-table [id=5e6cbe0169474e3aac421d59125936f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:36.508813 14478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88ad5c98350d4c44949e5947a0312af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:36.515834 14597 tablet_bootstrap.cc:492] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: Bootstrap starting.
I20260806 16:06:36.516453 14596 tablet_bootstrap.cc:492] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7: Bootstrap starting.
I20260806 16:06:36.517629 14598 tablet_bootstrap.cc:492] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: Bootstrap starting.
I20260806 16:06:36.520277 14598 tablet_bootstrap.cc:654] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:36.521211 14597 tablet_bootstrap.cc:654] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:36.522058 14598 log.cc:824] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:36.522557 14597 log.cc:824] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:36.523707 14596 tablet_bootstrap.cc:654] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:36.524626 14596 log.cc:824] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:36.525306 14597 tablet_bootstrap.cc:492] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: No bootstrap required, opened a new log
I20260806 16:06:36.525290 14598 tablet_bootstrap.cc:492] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: No bootstrap required, opened a new log
I20260806 16:06:36.525626 14597 ts_tablet_manager.cc:1397] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260806 16:06:36.525898 14598 ts_tablet_manager.cc:1397] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:06:36.527063 14596 tablet_bootstrap.cc:492] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7: No bootstrap required, opened a new log
I20260806 16:06:36.527280 14596 ts_tablet_manager.cc:1397] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260806 16:06:36.529826 14598 raft_consensus.cc:348] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17327227f08848b99763a79987bacaf6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43281 } } peers { permanent_uuid: "c8c32242398f4de08c0401d0e4122eb7" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34661 } } peers { permanent_uuid: "3a7486e882f14306bdf25aa9a197ce19" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 33603 } }
I20260806 16:06:36.530145 14598 raft_consensus.cc:374] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:36.530202 14598 raft_consensus.cc:729] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17327227f08848b99763a79987bacaf6, State: Initialized, Role: FOLLOWER
I20260806 16:06:36.530432 14597 raft_consensus.cc:348] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17327227f08848b99763a79987bacaf6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43281 } } peers { permanent_uuid: "c8c32242398f4de08c0401d0e4122eb7" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34661 } } peers { permanent_uuid: "3a7486e882f14306bdf25aa9a197ce19" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 33603 } }
I20260806 16:06:36.530786 14597 raft_consensus.cc:374] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:36.530707 14598 consensus_queue.cc:260] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17327227f08848b99763a79987bacaf6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43281 } } peers { permanent_uuid: "c8c32242398f4de08c0401d0e4122eb7" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34661 } } peers { permanent_uuid: "3a7486e882f14306bdf25aa9a197ce19" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 33603 } }
I20260806 16:06:36.530885 14597 raft_consensus.cc:729] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7486e882f14306bdf25aa9a197ce19, State: Initialized, Role: FOLLOWER
I20260806 16:06:36.531487 14597 consensus_queue.cc:260] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17327227f08848b99763a79987bacaf6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43281 } } peers { permanent_uuid: "c8c32242398f4de08c0401d0e4122eb7" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34661 } } peers { permanent_uuid: "3a7486e882f14306bdf25aa9a197ce19" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 33603 } }
I20260806 16:06:36.531661 14570 heartbeater.cc:503] Master 127.8.49.62:46219 was elected leader, sending a full tablet report...
I20260806 16:06:36.532020 14598 ts_tablet_manager.cc:1428] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 16:06:36.532423 14383 heartbeater.cc:503] Master 127.8.49.62:46219 was elected leader, sending a full tablet report...
I20260806 16:06:36.532686 14597 ts_tablet_manager.cc:1428] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:06:36.533273 14596 raft_consensus.cc:348] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17327227f08848b99763a79987bacaf6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43281 } } peers { permanent_uuid: "c8c32242398f4de08c0401d0e4122eb7" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34661 } } peers { permanent_uuid: "3a7486e882f14306bdf25aa9a197ce19" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 33603 } }
I20260806 16:06:36.533615 14596 raft_consensus.cc:374] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:36.533672 14596 raft_consensus.cc:729] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8c32242398f4de08c0401d0e4122eb7, State: Initialized, Role: FOLLOWER
I20260806 16:06:36.534158 14596 consensus_queue.cc:260] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17327227f08848b99763a79987bacaf6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43281 } } peers { permanent_uuid: "c8c32242398f4de08c0401d0e4122eb7" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34661 } } peers { permanent_uuid: "3a7486e882f14306bdf25aa9a197ce19" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 33603 } }
I20260806 16:06:36.535405 14191 heartbeater.cc:503] Master 127.8.49.62:46219 was elected leader, sending a full tablet report...
I20260806 16:06:36.535406 14596 ts_tablet_manager.cc:1428] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260806 16:06:36.585700 14192 tablet.cc:2369] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:06:36.624894 14384 tablet.cc:2369] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:06:36.677983 14571 tablet.cc:2369] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:36.753647 14605 raft_consensus.cc:482] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:36.754109 14605 raft_consensus.cc:504] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17327227f08848b99763a79987bacaf6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43281 } } peers { permanent_uuid: "c8c32242398f4de08c0401d0e4122eb7" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34661 } } peers { permanent_uuid: "3a7486e882f14306bdf25aa9a197ce19" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 33603 } }
I20260806 16:06:36.755901 14605 leader_election.cc:288] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17327227f08848b99763a79987bacaf6 (127.8.49.3:43281), 3a7486e882f14306bdf25aa9a197ce19 (127.8.49.2:33603)
I20260806 16:06:36.762135 14607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.756445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:43281 (local address 127.8.49.1:58593)
0806 16:06:36.757089 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:36.757104 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:36.757322 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:36.757631 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:36.757636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:36.757658 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:36.757885 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.757892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.758914 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.758919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.760421 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.760434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.761475 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.761480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.761615 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.761654 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 16:06:36.761765 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 16:06:36.761876 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":112,"spinlock_wait_cycles":58624,"thread_start_us":106,"threads_started":1}
I20260806 16:06:36.762224 14591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.756649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:58593 (local address 127.8.49.3:43281)
0806 16:06:36.756835 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:36.756841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:36.757329 (+ 488us) server_negotiation.cc:407] Connection header received
0806 16:06:36.757427 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:36.757433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:36.757511 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:36.757617 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:36.758025 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.758695 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.760589 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.761209 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.761409 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.761495 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:36.761818 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 16:06:36.761920 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:06:36.762445 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.756746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:39789 (local address 127.8.49.2:33603)
0806 16:06:36.756903 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:36.756911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:36.757202 (+ 291us) server_negotiation.cc:407] Connection header received
0806 16:06:36.757505 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:36.757508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:36.757580 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:36.757685 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:36.758252 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.759082 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.760693 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.761473 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.761623 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.761713 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:36.762167 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 16:06:36.762283 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:06:36.762575 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.756357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:33603 (local address 127.8.49.1:39789)
0806 16:06:36.757010 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:36.757048 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:36.757302 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:36.757715 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:36.757718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:36.757788 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:36.757997 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.758009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.759249 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.759252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.760438 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:36.760453 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.761713 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.761717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:36.761896 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.761931 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 16:06:36.762149 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 16:06:36.762266 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":3,"spinlock_wait_cycles":1280,"thread_start_us":157,"threads_started":1}
I20260806 16:06:36.763659 14319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ad5c98350d4c44949e5947a0312af0" candidate_uuid: "c8c32242398f4de08c0401d0e4122eb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7486e882f14306bdf25aa9a197ce19" is_pre_election: true
I20260806 16:06:36.763679 14501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ad5c98350d4c44949e5947a0312af0" candidate_uuid: "c8c32242398f4de08c0401d0e4122eb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17327227f08848b99763a79987bacaf6" is_pre_election: true
I20260806 16:06:36.764145 14501 raft_consensus.cc:2454] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8c32242398f4de08c0401d0e4122eb7 in term 0.
I20260806 16:06:36.764158 14319 raft_consensus.cc:2454] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8c32242398f4de08c0401d0e4122eb7 in term 0.
I20260806 16:06:36.764891 14032 leader_election.cc:302] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 17327227f08848b99763a79987bacaf6, 3a7486e882f14306bdf25aa9a197ce19, c8c32242398f4de08c0401d0e4122eb7; no voters:
I20260806 16:06:36.765323 14605 raft_consensus.cc:2781] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:36.765455 14605 raft_consensus.cc:482] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:36.765520 14605 raft_consensus.cc:3037] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:36.766808 14605 raft_consensus.cc:504] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17327227f08848b99763a79987bacaf6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43281 } } peers { permanent_uuid: "c8c32242398f4de08c0401d0e4122eb7" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34661 } } peers { permanent_uuid: "3a7486e882f14306bdf25aa9a197ce19" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 33603 } }
I20260806 16:06:36.767354 14605 leader_election.cc:288] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [CANDIDATE]: Term 1 election: Requested vote from peers 17327227f08848b99763a79987bacaf6 (127.8.49.3:43281), 3a7486e882f14306bdf25aa9a197ce19 (127.8.49.2:33603)
I20260806 16:06:36.767794 14319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ad5c98350d4c44949e5947a0312af0" candidate_uuid: "c8c32242398f4de08c0401d0e4122eb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7486e882f14306bdf25aa9a197ce19"
I20260806 16:06:36.767925 14319 raft_consensus.cc:3037] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:36.767999 14501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ad5c98350d4c44949e5947a0312af0" candidate_uuid: "c8c32242398f4de08c0401d0e4122eb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17327227f08848b99763a79987bacaf6"
I20260806 16:06:36.768225 14501 raft_consensus.cc:3037] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:36.769078 14319 raft_consensus.cc:2454] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8c32242398f4de08c0401d0e4122eb7 in term 1.
I20260806 16:06:36.769518 14033 leader_election.cc:302] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a7486e882f14306bdf25aa9a197ce19, c8c32242398f4de08c0401d0e4122eb7; no voters:
I20260806 16:06:36.769762 14605 raft_consensus.cc:2781] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:36.770049 14605 raft_consensus.cc:686] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [term 1 LEADER]: Becoming Leader. State: Replica: c8c32242398f4de08c0401d0e4122eb7, State: Running, Role: LEADER
I20260806 16:06:36.770104 14501 raft_consensus.cc:2454] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8c32242398f4de08c0401d0e4122eb7 in term 1.
I20260806 16:06:36.770479 14605 consensus_queue.cc:237] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17327227f08848b99763a79987bacaf6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43281 } } peers { permanent_uuid: "c8c32242398f4de08c0401d0e4122eb7" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34661 } } peers { permanent_uuid: "3a7486e882f14306bdf25aa9a197ce19" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 33603 } }
I20260806 16:06:36.774385 13916 catalog_manager.cc:5495] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8c32242398f4de08c0401d0e4122eb7 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "c8c32242398f4de08c0401d0e4122eb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17327227f08848b99763a79987bacaf6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8c32242398f4de08c0401d0e4122eb7" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a7486e882f14306bdf25aa9a197ce19" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 33603 } health_report { overall_health: UNKNOWN } } }
I20260806 16:06:36.810168 14595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.804639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.8.49.1:34661)
0806 16:06:36.804918 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:36.804923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:36.804945 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:06:36.805074 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:36.805077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:36.805163 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:36.805320 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:36.805828 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.806447 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.807772 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:36.808039 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:36.808165 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:36.808631 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:36.809291 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:36.809592 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:36.809752 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 16:06:36.809903 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 16:06:36.819643 14319 raft_consensus.cc:1261] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19 [term 1 FOLLOWER]: Refusing update from remote peer c8c32242398f4de08c0401d0e4122eb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:36.820041 14501 raft_consensus.cc:1261] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6 [term 1 FOLLOWER]: Refusing update from remote peer c8c32242398f4de08c0401d0e4122eb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:36.820667 14605 consensus_queue.cc:1035] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a7486e882f14306bdf25aa9a197ce19" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 33603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:36.821429 14610 consensus_queue.cc:1035] T 88ad5c98350d4c44949e5947a0312af0 P c8c32242398f4de08c0401d0e4122eb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17327227f08848b99763a79987bacaf6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:36.830920 14615 mvcc.cc:205] Tried to move back new op lower bound from 7315588697361592320 to 7315588697175494656. Current Snapshot: MvccSnapshot[applied={T|T < 7315588697361592320}]
I20260806 16:06:36.831009 14616 mvcc.cc:205] Tried to move back new op lower bound from 7315588697361592320 to 7315588697175494656. Current Snapshot: MvccSnapshot[applied={T|T < 7315588697361592320}]
I20260806 16:06:36.834180 14617 mvcc.cc:205] Tried to move back new op lower bound from 7315588697361592320 to 7315588697175494656. Current Snapshot: MvccSnapshot[applied={T|T < 7315588697361592320}]
I20260806 16:06:36.848914 13917 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:36.849253 13917 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:36.853579 13917 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7bcc606c6c69462392fd58eb2b843cda: Sending DeleteTablet for 3 replicas of tablet 88ad5c98350d4c44949e5947a0312af0
I20260806 16:06:36.855324 14094 tablet_service.cc:1514] Processing DeleteTablet for tablet 88ad5c98350d4c44949e5947a0312af0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:36 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:36144
I20260806 16:06:36.855317 14478 tablet_service.cc:1514] Processing DeleteTablet for tablet 88ad5c98350d4c44949e5947a0312af0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:36 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:51838
I20260806 16:06:36.855357 14283 tablet_service.cc:1514] Processing DeleteTablet for tablet 88ad5c98350d4c44949e5947a0312af0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:36 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:34906
I20260806 16:06:36.856292 14629 tablet_replica.cc:330] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: stopping tablet replica
I20260806 16:06:36.856292 14630 tablet_replica.cc:330] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: stopping tablet replica
I20260806 16:06:36.856626 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 14009
I20260806 16:06:36.857203 14630 raft_consensus.cc:2229] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:36.858047 14630 raft_consensus.cc:2258] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:36.861120 14629 raft_consensus.cc:2229] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:36.861994 14629 raft_consensus.cc:2258] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:36.863511 14630 ts_tablet_manager.cc:1905] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:36.866650 14629 ts_tablet_manager.cc:1905] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:36.867856 14630 ts_tablet_manager.cc:1918] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:36.868055 14630 log.cc:1196] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/88ad5c98350d4c44949e5947a0312af0
I20260806 16:06:36.868491 14630 ts_tablet_manager.cc:1939] T 88ad5c98350d4c44949e5947a0312af0 P 17327227f08848b99763a79987bacaf6: Deleting consensus metadata
I20260806 16:06:36.870612 13897 catalog_manager.cc:4841] TS 17327227f08848b99763a79987bacaf6 (127.8.49.3:43281): tablet 88ad5c98350d4c44949e5947a0312af0 (table test-table [id=5e6cbe0169474e3aac421d59125936f0]) successfully deleted
I20260806 16:06:36.871145 14629 ts_tablet_manager.cc:1918] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:36.871315 14629 log.cc:1196] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/88ad5c98350d4c44949e5947a0312af0
I20260806 16:06:36.871726 14629 ts_tablet_manager.cc:1939] T 88ad5c98350d4c44949e5947a0312af0 P 3a7486e882f14306bdf25aa9a197ce19: Deleting consensus metadata
I20260806 16:06:36.873688 13898 catalog_manager.cc:4841] TS 3a7486e882f14306bdf25aa9a197ce19 (127.8.49.2:33603): tablet 88ad5c98350d4c44949e5947a0312af0 (table test-table [id=5e6cbe0169474e3aac421d59125936f0]) successfully deleted
W20260806 16:06:36.874531 13897 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:34661 (error 108)
I20260806 16:06:36.874832 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 14200
I20260806 16:06:36.878099 14592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:36.877600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:34661 (local address 127.0.0.1:36154)
0806 16:06:36.877870 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:36.877976 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260806 16:06:36.878625 13897 catalog_manager.cc:4584] TS c8c32242398f4de08c0401d0e4122eb7 (127.8.49.1:34661): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88ad5c98350d4c44949e5947a0312af0: Network error: Client connection negotiation failed: client connection to 127.8.49.1:34661: connect: Connection refused (error 111)
I20260806 16:06:36.889530 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 14389
I20260806 16:06:36.905093 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 13875
2026-08-06T16:06:36Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3289 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T16:06:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:36Z Disabled control of system clock
I20260806 16:06:36.955396 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:37023
--webserver_interface=127.8.49.62
--webserver_port=0
--builtin_ntp_servers=127.8.49.20:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:37023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 16:06:37.097853 14637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:37.098225 14637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:37.098270 14637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:37.103380 14637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:37.103555 14637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:37.103578 14637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:37.103597 14637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:37.103614 14637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:37.109072 14637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:37023
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:37023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:37.111259 14637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:37.113246 14637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:37.125304 14648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:37.125432 14637 server_base.cc:1034] running on GCE node
W20260806 16:06:37.125310 14651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:37.125655 14647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:37.126657 14637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:37.128551 14637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:37.130059 14637 hybrid_clock.cc:648] HybridClock initialized: now 1786032397129997 us; error 73 us; skew 500 ppm
I20260806 16:06:37.133370 14637 webserver.cc:460] Webserver started at http://127.8.49.62:39635/ using document root <none> and password file <none>
I20260806 16:06:37.134214 14637 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:37.134297 14637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:37.134588 14637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:37.137567 14637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "e11f162fd68b42149b4a43cec82d294a"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "24d1da565bf823365da347ae5f29e4e2"
server_key_iv: "bb4a677f71c90f53320c23301b119b89"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.138365 14637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "e11f162fd68b42149b4a43cec82d294a"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "24d1da565bf823365da347ae5f29e4e2"
server_key_iv: "bb4a677f71c90f53320c23301b119b89"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.143597 14637 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:06:37.147293 14660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:37.149102 14637 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:37.149300 14637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "e11f162fd68b42149b4a43cec82d294a"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "24d1da565bf823365da347ae5f29e4e2"
server_key_iv: "bb4a677f71c90f53320c23301b119b89"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.149461 14637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:37.155290 14637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:37.156044 14637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:37.156265 14637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:37.165400 14637 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:37023
I20260806 16:06:37.165421 14733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:37023 every 8 connection(s)
I20260806 16:06:37.166563 14637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:37.171700 14734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:37.175827 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 14637
I20260806 16:06:37.175999 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:37.176331 8388 external_mini_cluster.cc:1428] Setting key 0efbf07c71d2091c77896d847503cec8
I20260806 16:06:37.181121 14734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a: Bootstrap starting.
I20260806 16:06:37.185220 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:37.178477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43486 (local address 127.8.49.62:37023)
0806 16:06:37.179628 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:37.179658 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:37.179723 (+ 65us) server_negotiation.cc:407] Connection header received
0806 16:06:37.180809 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:37.180841 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:37.181494 (+ 653us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:37.182205 (+ 711us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:37.183262 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:37.183269 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:37.183286 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:37.183301 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:37.183393 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:37.183423 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:37.183625 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:37.183714 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 16:06:37.184033 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":280,"threads_started":1}
I20260806 16:06:37.185276 14734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:37.186895 14734 log.cc:824] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:37.189988 14734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a: No bootstrap required, opened a new log
I20260806 16:06:37.195147 14734 raft_consensus.cc:348] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11f162fd68b42149b4a43cec82d294a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37023 } }
I20260806 16:06:37.195844 14734 raft_consensus.cc:374] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:37.195930 14734 raft_consensus.cc:729] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e11f162fd68b42149b4a43cec82d294a, State: Initialized, Role: FOLLOWER
I20260806 16:06:37.196501 14734 consensus_queue.cc:260] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11f162fd68b42149b4a43cec82d294a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37023 } }
I20260806 16:06:37.196664 14734 raft_consensus.cc:388] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:37.196723 14734 raft_consensus.cc:482] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:37.196822 14734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:37.198115 14734 raft_consensus.cc:504] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11f162fd68b42149b4a43cec82d294a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37023 } }
I20260806 16:06:37.198563 14734 leader_election.cc:302] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e11f162fd68b42149b4a43cec82d294a; no voters:
I20260806 16:06:37.199102 14734 leader_election.cc:288] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:37.199304 14740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:37.199713 14740 raft_consensus.cc:686] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [term 1 LEADER]: Becoming Leader. State: Replica: e11f162fd68b42149b4a43cec82d294a, State: Running, Role: LEADER
I20260806 16:06:37.200160 14740 consensus_queue.cc:237] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11f162fd68b42149b4a43cec82d294a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37023 } }
I20260806 16:06:37.201051 14734 sys_catalog.cc:564] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:37.202636 14743 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e11f162fd68b42149b4a43cec82d294a. Latest consensus state: current_term: 1 leader_uuid: "e11f162fd68b42149b4a43cec82d294a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11f162fd68b42149b4a43cec82d294a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37023 } } }
I20260806 16:06:37.202636 14741 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e11f162fd68b42149b4a43cec82d294a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11f162fd68b42149b4a43cec82d294a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37023 } } }
I20260806 16:06:37.203015 14741 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:37.203413 14743 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:37.203560 14751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:37.207732 14751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:37.215948 14751 catalog_manager.cc:1347] Generated new cluster ID: 2ef52f28c3a54913be6b4a37ffedb1b6
I20260806 16:06:37.216058 14751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:37.231192 14751 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:37.231608 14751 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:37.241025 14751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a: Generated new TSK 0
I20260806 16:06:37.241951 14751 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:06:37.245443 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37023
--builtin_ntp_servers=127.8.49.20:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 16:06:37.382530 14762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:37.383010 14762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:37.383116 14762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:37.388116 14762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:37.388259 14762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:37.388378 14762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:37.394586 14762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:37.395946 14762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:37.397497 14762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:37.409999 14768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:37.410077 14769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:37.410076 14771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:37.410709 14762 server_base.cc:1034] running on GCE node
I20260806 16:06:37.411232 14762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:37.412004 14762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:37.413228 14762 hybrid_clock.cc:648] HybridClock initialized: now 1786032397413173 us; error 78 us; skew 500 ppm
I20260806 16:06:37.415655 14762 webserver.cc:460] Webserver started at http://127.8.49.1:39939/ using document root <none> and password file <none>
I20260806 16:06:37.416338 14762 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:37.416441 14762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:37.416702 14762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:37.419137 14762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "1f3f3486bbb1406a8bcd1378bc5ce700"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "22235b53232614e7744920d93d712e2e"
server_key_iv: "9f5667bb533f1cbb3a401e079792341e"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.419893 14762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "1f3f3486bbb1406a8bcd1378bc5ce700"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "22235b53232614e7744920d93d712e2e"
server_key_iv: "9f5667bb533f1cbb3a401e079792341e"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.424547 14762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:37.427995 14780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:37.429909 14762 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:37.430135 14762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "1f3f3486bbb1406a8bcd1378bc5ce700"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "22235b53232614e7744920d93d712e2e"
server_key_iv: "9f5667bb533f1cbb3a401e079792341e"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.430294 14762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:37.435901 14762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:37.436628 14762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:37.436848 14762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:37.437518 14762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:37.438644 14762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:37.438728 14762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:37.438889 14762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:37.438943 14762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 16:06:37.451260 14762 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:32841
I20260806 16:06:37.451364 14943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:32841 every 8 connection(s)
I20260806 16:06:37.452435 14762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:37.455647 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 14762
I20260806 16:06:37.455833 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:37.456221 8388 external_mini_cluster.cc:1428] Setting key 08097179090c3ecd5e630af3175b0404
I20260806 16:06:37.459263 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37023
--builtin_ntp_servers=127.8.49.20:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 16:06:37.462782 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:37.455865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:41897 (local address 127.8.49.62:37023)
0806 16:06:37.456050 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:37.456054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:37.458010 (+ 1956us) server_negotiation.cc:407] Connection header received
0806 16:06:37.460042 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:37.460046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:37.460114 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:37.460240 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:37.462066 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:37.462073 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:37.462078 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:37.462100 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:37.462135 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:37.462149 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:37.462298 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:37.462488 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 16:06:37.462558 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 16:06:37.464133 14947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:37.456679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37023 (local address 127.8.49.1:41897)
0806 16:06:37.457768 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:37.457838 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:37.459726 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:37.460473 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:37.460482 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:37.460603 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:37.461587 (+ 984us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:06:37.461590 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:37.461664 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:06:37.461803 (+ 139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:37.462293 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:37.462304 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:37.462310 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 16:06:37.462514 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 16:06:37.462941 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":221,"threads_started":1}
I20260806 16:06:37.465871 14944 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37023
I20260806 16:06:37.466321 14944 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:37.467140 14944 heartbeater.cc:511] Master 127.8.49.62:37023 requested a full tablet report, sending...
I20260806 16:06:37.469436 14684 ts_manager.cc:194] Registered new tserver with Master: 1f3f3486bbb1406a8bcd1378bc5ce700 (127.8.49.1:32841)
W20260806 16:06:37.608827 14948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:37.609253 14948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:37.609345 14948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:37.614311 14948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:37.614478 14948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:37.614621 14948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:37.621738 14948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:37.623917 14948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:37.626024 14948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:37.640198 14959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:37.640198 14961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:37.640203 14958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:37.640954 14948 server_base.cc:1034] running on GCE node
I20260806 16:06:37.641566 14948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:37.642560 14948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:37.643807 14948 hybrid_clock.cc:648] HybridClock initialized: now 1786032397643774 us; error 58 us; skew 500 ppm
I20260806 16:06:37.647183 14948 webserver.cc:460] Webserver started at http://127.8.49.2:37221/ using document root <none> and password file <none>
I20260806 16:06:37.648090 14948 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:37.648207 14948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:37.648712 14948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:37.651885 14948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "65bf4208c7e74528a9d469dddd42cba3"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "a6e4cd151eba2a67fc1933972c94c139"
server_key_iv: "fffe5faa676cd0759ba16649671d1749"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.652673 14948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "65bf4208c7e74528a9d469dddd42cba3"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "a6e4cd151eba2a67fc1933972c94c139"
server_key_iv: "fffe5faa676cd0759ba16649671d1749"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.658469 14948 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:06:37.663056 14973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:37.665369 14948 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:37.665643 14948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "65bf4208c7e74528a9d469dddd42cba3"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "a6e4cd151eba2a67fc1933972c94c139"
server_key_iv: "fffe5faa676cd0759ba16649671d1749"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.665861 14948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:37.672925 14948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:37.673983 14948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:37.674264 14948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:37.675218 14948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:37.676611 14948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:37.676698 14948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:37.676800 14948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:37.676862 14948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:37.691318 14948 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:38793
I20260806 16:06:37.691361 15138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:38793 every 8 connection(s)
I20260806 16:06:37.692763 14948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:37.700613 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 14948
I20260806 16:06:37.700811 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:37.701236 8388 external_mini_cluster.cc:1428] Setting key 8ccee73f3490004dd63319bd06beeb13
I20260806 16:06:37.702958 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:37.695751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:36239 (local address 127.8.49.62:37023)
0806 16:06:37.695991 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:37.695996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:37.697647 (+ 1651us) server_negotiation.cc:407] Connection header received
0806 16:06:37.699347 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:37.699358 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:37.699431 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:37.699558 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:37.701922 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:37.701928 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:37.701932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:37.701961 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:37.702003 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:37.702018 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:37.702182 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:37.702686 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 16:06:37.702759 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 16:06:37.704842 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37023
--builtin_ntp_servers=127.8.49.20:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 16:06:37.704855 15142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:37.696329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37023 (local address 127.8.49.2:36239)
0806 16:06:37.697358 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:37.697421 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:37.698966 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:37.699848 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:37.699858 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:37.699971 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:37.701197 (+ 1226us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:06:37.701199 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:37.701274 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:06:37.701404 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:37.702340 (+ 936us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:37.702370 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:37.702379 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:06:37.702843 (+ 464us) client_negotiation.cc:240] Negotiation successful
0806 16:06:37.703364 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":205,"threads_started":1}
I20260806 16:06:37.707512 15139 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37023
I20260806 16:06:37.707939 15139 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:37.708887 15139 heartbeater.cc:511] Master 127.8.49.62:37023 requested a full tablet report, sending...
I20260806 16:06:37.711714 14684 ts_manager.cc:194] Registered new tserver with Master: 65bf4208c7e74528a9d469dddd42cba3 (127.8.49.2:38793)
W20260806 16:06:37.866787 15143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:37.867244 15143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:37.867296 15143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:37.872296 15143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:37.872440 15143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:37.872514 15143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:37.878206 15143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:37.880147 15143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:37.882179 15143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:37.895433 15160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:37.895421 15157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:37.895676 15156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:37.896034 15143 server_base.cc:1034] running on GCE node
I20260806 16:06:37.896958 15143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:37.898065 15143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:37.899323 15143 hybrid_clock.cc:648] HybridClock initialized: now 1786032397899321 us; error 80 us; skew 500 ppm
I20260806 16:06:37.908288 15143 webserver.cc:460] Webserver started at http://127.8.49.3:41237/ using document root <none> and password file <none>
I20260806 16:06:37.909170 15143 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:37.909268 15143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:37.909559 15143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:37.912436 15143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "f76beacd885e4f73a0877c49b6a6b5bb"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "a011626e1adee710b8e31ee45e4508c4"
server_key_iv: "d83fd0c388c840bb14c94f31bb98c9e5"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.913106 15143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "f76beacd885e4f73a0877c49b6a6b5bb"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "a011626e1adee710b8e31ee45e4508c4"
server_key_iv: "d83fd0c388c840bb14c94f31bb98c9e5"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.918622 15143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:06:37.922525 15169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:37.924559 15143 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:37.924799 15143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "f76beacd885e4f73a0877c49b6a6b5bb"
format_stamp: "Formatted at 2026-08-06 16:06:37 on dist-test-slave-7z32"
server_key: "a011626e1adee710b8e31ee45e4508c4"
server_key_iv: "d83fd0c388c840bb14c94f31bb98c9e5"
server_key_version: "encryptionkey@0"
I20260806 16:06:37.924988 15143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:37.931110 15143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:37.931923 15143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:37.932276 15143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:37.933120 15143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:37.934279 15143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:37.934336 15143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:37.934381 15143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:37.934553 15143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 16:06:37.946203 15143 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:35617
I20260806 16:06:37.946378 15321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:35617 every 8 connection(s)
I20260806 16:06:37.947474 15143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:37.955853 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:37.950271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:41441 (local address 127.8.49.62:37023)
0806 16:06:37.950460 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:37.950466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:37.952050 (+ 1584us) server_negotiation.cc:407] Connection header received
0806 16:06:37.953569 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:37.953572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:37.953626 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:37.953724 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:37.955324 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:37.955328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:37.955331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:37.955351 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:37.955373 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:37.955381 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:37.955485 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:37.955650 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 16:06:37.955702 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:06:37.956913 15327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:37.950802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37023 (local address 127.8.49.3:41441)
0806 16:06:37.951814 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:37.951869 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:37.953257 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:37.953924 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:37.953932 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:37.954028 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:37.955035 (+ 1007us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:06:37.955037 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:37.955081 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:06:37.955158 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:37.955486 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:37.955499 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:37.955507 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 16:06:37.955705 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 16:06:37.955983 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":166,"threads_started":1}
I20260806 16:06:37.957767 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 15143
I20260806 16:06:37.957929 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:37.958340 8388 external_mini_cluster.cc:1428] Setting key 8a3b484430f4cd3a92c934ce746f22ee
I20260806 16:06:37.958814 15322 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37023
I20260806 16:06:37.959203 15322 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:37.959699 15322 heartbeater.cc:511] Master 127.8.49.62:37023 requested a full tablet report, sending...
I20260806 16:06:37.961269 14678 ts_manager.cc:194] Registered new tserver with Master: f76beacd885e4f73a0877c49b6a6b5bb (127.8.49.3:35617)
I20260806 16:06:37.974416 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:06:37.979532 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:37.976886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.8.49.62:37023)
0806 16:06:37.977198 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:37.977233 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:37.977290 (+ 57us) server_negotiation.cc:407] Connection header received
0806 16:06:37.977605 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:37.977612 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:37.977669 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:37.977787 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:37.978928 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:37.978933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:37.978938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:37.978954 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:37.978984 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:37.978997 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:37.979120 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:37.979275 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 16:06:37.979329 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260806 16:06:37.984579 14678 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:37.989084 14678 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:38.006791 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.003511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39056 (local address 127.8.49.3:35617)
0806 16:06:38.004037 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:38.004053 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:38.004087 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:06:38.004162 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:38.004167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:38.004371 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:38.004575 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:38.005917 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:38.005927 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:38.005946 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:38.005961 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:38.006019 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:38.006057 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:38.006155 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:38.006435 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 16:06:38.006545 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":277,"threads_started":1}
I20260806 16:06:38.007479 15337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.003233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:35617 (local address 127.0.0.1:39056)
0806 16:06:38.003754 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:38.003829 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:38.003976 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:38.004603 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:38.004611 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:38.004708 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:38.005635 (+ 927us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:06:38.005637 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:38.005661 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:06:38.005744 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:38.006246 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:38.006257 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:38.006266 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:06:38.007199 (+ 933us) client_negotiation.cc:240] Negotiation successful
0806 16:06:38.007296 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":190,"threads_started":1}
I20260806 16:06:38.009572 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.005188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:32841 (local address 127.0.0.1:42306)
0806 16:06:38.005716 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:38.005733 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:38.005829 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:38.008042 (+ 2213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:38.008046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:38.008067 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:38.008777 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:06:38.008780 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:38.008799 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:06:38.008840 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:38.009275 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:38.009278 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:38.009281 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 16:06:38.009376 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 16:06:38.009417 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":188,"threads_started":1}
I20260806 16:06:38.009869 15337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.006797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:38793 (local address 127.0.0.1:45414)
0806 16:06:38.007766 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:38.007783 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:38.007864 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:38.008577 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:38.008581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:38.008598 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:38.009017 (+ 419us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:06:38.009021 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:38.009037 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:06:38.009071 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:38.009619 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:38.009625 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:38.009629 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 16:06:38.009711 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 16:06:38.009747 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":160,"threads_started":1}
I20260806 16:06:38.009727 15341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.005800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.8.49.1:32841)
0806 16:06:38.006505 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:38.006517 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:38.006554 (+ 37us) server_negotiation.cc:407] Connection header received
0806 16:06:38.006639 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:38.006647 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:38.006967 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:38.007168 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:38.009027 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:38.009036 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:38.009052 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:38.009067 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:38.009120 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:38.009164 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:38.009301 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:38.009411 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 16:06:38.009518 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":171,"threads_started":1}
I20260806 16:06:38.010356 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.007610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45414 (local address 127.8.49.2:38793)
0806 16:06:38.008105 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:38.008119 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:38.008150 (+ 31us) server_negotiation.cc:407] Connection header received
0806 16:06:38.008212 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:38.008219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:38.008424 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:38.008590 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:38.009363 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:38.009370 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:38.009384 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:38.009395 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:38.009437 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:38.009464 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:38.009571 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:38.010082 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 16:06:38.010165 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":167,"threads_started":1}
I20260806 16:06:38.013209 15227 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8240f43f57456ea9259ccab27654ee (DEFAULT_TABLE table=test-table [id=3f773e09ffdd42488d2506a47ed3a9fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:38.013602 14857 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8240f43f57456ea9259ccab27654ee (DEFAULT_TABLE table=test-table [id=3f773e09ffdd42488d2506a47ed3a9fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:38.014401 15038 tablet_service.cc:1467] Processing CreateTablet for tablet 4c8240f43f57456ea9259ccab27654ee (DEFAULT_TABLE table=test-table [id=3f773e09ffdd42488d2506a47ed3a9fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:38.014962 15227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8240f43f57456ea9259ccab27654ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:38.015204 14857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8240f43f57456ea9259ccab27654ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:38.015520 15038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c8240f43f57456ea9259ccab27654ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:38.025133 15345 tablet_bootstrap.cc:492] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: Bootstrap starting.
I20260806 16:06:38.027181 15347 tablet_bootstrap.cc:492] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700: Bootstrap starting.
I20260806 16:06:38.028059 15345 tablet_bootstrap.cc:654] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:38.029201 15345 log.cc:824] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:38.029858 15348 tablet_bootstrap.cc:492] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: Bootstrap starting.
I20260806 16:06:38.030442 15347 tablet_bootstrap.cc:654] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:38.031632 15347 log.cc:824] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:38.032748 15345 tablet_bootstrap.cc:492] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: No bootstrap required, opened a new log
I20260806 16:06:38.033092 15345 ts_tablet_manager.cc:1397] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:06:38.033586 15348 tablet_bootstrap.cc:654] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:38.035322 15348 log.cc:824] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:38.038187 15347 tablet_bootstrap.cc:492] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700: No bootstrap required, opened a new log
I20260806 16:06:38.038501 15347 ts_tablet_manager.cc:1397] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260806 16:06:38.038950 15348 tablet_bootstrap.cc:492] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: No bootstrap required, opened a new log
I20260806 16:06:38.039144 15348 ts_tablet_manager.cc:1397] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 16:06:38.039841 15345 raft_consensus.cc:348] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35617 } } peers { permanent_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 32841 } } peers { permanent_uuid: "65bf4208c7e74528a9d469dddd42cba3" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 38793 } }
I20260806 16:06:38.040261 15345 raft_consensus.cc:374] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:38.040325 15345 raft_consensus.cc:729] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76beacd885e4f73a0877c49b6a6b5bb, State: Initialized, Role: FOLLOWER
I20260806 16:06:38.041002 15345 consensus_queue.cc:260] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35617 } } peers { permanent_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 32841 } } peers { permanent_uuid: "65bf4208c7e74528a9d469dddd42cba3" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 38793 } }
I20260806 16:06:38.041864 15322 heartbeater.cc:503] Master 127.8.49.62:37023 was elected leader, sending a full tablet report...
I20260806 16:06:38.044739 15348 raft_consensus.cc:348] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35617 } } peers { permanent_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 32841 } } peers { permanent_uuid: "65bf4208c7e74528a9d469dddd42cba3" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 38793 } }
I20260806 16:06:38.045341 15348 raft_consensus.cc:374] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:38.045447 15348 raft_consensus.cc:729] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65bf4208c7e74528a9d469dddd42cba3, State: Initialized, Role: FOLLOWER
I20260806 16:06:38.046192 15348 consensus_queue.cc:260] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35617 } } peers { permanent_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 32841 } } peers { permanent_uuid: "65bf4208c7e74528a9d469dddd42cba3" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 38793 } }
I20260806 16:06:38.046254 15347 raft_consensus.cc:348] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35617 } } peers { permanent_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 32841 } } peers { permanent_uuid: "65bf4208c7e74528a9d469dddd42cba3" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 38793 } }
I20260806 16:06:38.046634 15347 raft_consensus.cc:374] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:38.046741 15347 raft_consensus.cc:729] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3f3486bbb1406a8bcd1378bc5ce700, State: Initialized, Role: FOLLOWER
I20260806 16:06:38.047111 15345 ts_tablet_manager.cc:1428] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: Time spent starting tablet: real 0.014s user 0.005s sys 0.004s
I20260806 16:06:38.047364 15139 heartbeater.cc:503] Master 127.8.49.62:37023 was elected leader, sending a full tablet report...
I20260806 16:06:38.047336 15347 consensus_queue.cc:260] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35617 } } peers { permanent_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 32841 } } peers { permanent_uuid: "65bf4208c7e74528a9d469dddd42cba3" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 38793 } }
I20260806 16:06:38.048058 15348 ts_tablet_manager.cc:1428] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260806 16:06:38.048825 14944 heartbeater.cc:503] Master 127.8.49.62:37023 was elected leader, sending a full tablet report...
I20260806 16:06:38.049031 15347 ts_tablet_manager.cc:1428] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700: Time spent starting tablet: real 0.010s user 0.002s sys 0.006s
I20260806 16:06:38.128007 15355 raft_consensus.cc:482] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:38.128309 15355 raft_consensus.cc:504] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35617 } } peers { permanent_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 32841 } } peers { permanent_uuid: "65bf4208c7e74528a9d469dddd42cba3" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 38793 } }
I20260806 16:06:38.130098 15355 leader_election.cc:288] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f76beacd885e4f73a0877c49b6a6b5bb (127.8.49.3:35617), 65bf4208c7e74528a9d469dddd42cba3 (127.8.49.2:38793)
I20260806 16:06:38.133276 15339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.130587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:59061 (local address 127.8.49.3:35617)
0806 16:06:38.130761 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:38.130766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:38.131076 (+ 310us) server_negotiation.cc:407] Connection header received
0806 16:06:38.131313 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:38.131317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:38.131368 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:38.131582 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:38.132774 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:38.132778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:38.132781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:38.132804 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:38.132825 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:38.132838 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:38.132957 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:38.133055 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 16:06:38.133111 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:06:38.133304 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.130387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:35617 (local address 127.8.49.1:59061)
0806 16:06:38.130961 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:38.130979 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:38.131176 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:38.131599 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:38.131602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:38.131630 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:38.132400 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:06:38.132403 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:38.132423 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:06:38.132470 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:38.132956 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:38.132959 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:38.132966 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:06:38.133065 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 16:06:38.133111 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":157,"threads_started":1}
I20260806 16:06:38.133761 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.130980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:36815 (local address 127.8.49.2:38793)
0806 16:06:38.131196 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:38.131203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:38.131718 (+ 515us) server_negotiation.cc:407] Connection header received
0806 16:06:38.131917 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:38.131921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:38.131978 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:38.132086 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:38.133035 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:38.133040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:38.133043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:38.133061 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:38.133097 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:38.133107 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:38.133221 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:38.133350 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 16:06:38.133540 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:06:38.134011 15358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.130788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:38793 (local address 127.8.49.1:36815)
0806 16:06:38.131559 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:38.131575 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:38.131784 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:38.132113 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:38.132116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:38.132132 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:38.132766 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:06:38.132768 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:38.132778 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:06:38.132806 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:38.133214 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:38.133218 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:38.133222 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 16:06:38.133334 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 16:06:38.133722 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":196,"threads_started":1}
I20260806 16:06:38.134363 15256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8240f43f57456ea9259ccab27654ee" candidate_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" is_pre_election: true
I20260806 16:06:38.134810 15256 raft_consensus.cc:2454] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f3f3486bbb1406a8bcd1378bc5ce700 in term 0.
I20260806 16:06:38.135033 15066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8240f43f57456ea9259ccab27654ee" candidate_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65bf4208c7e74528a9d469dddd42cba3" is_pre_election: true
I20260806 16:06:38.135357 15066 raft_consensus.cc:2454] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f3f3486bbb1406a8bcd1378bc5ce700 in term 0.
I20260806 16:06:38.135600 14781 leader_election.cc:302] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f3f3486bbb1406a8bcd1378bc5ce700, f76beacd885e4f73a0877c49b6a6b5bb; no voters:
I20260806 16:06:38.136077 15355 raft_consensus.cc:2781] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:38.136305 15355 raft_consensus.cc:482] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:38.136363 15355 raft_consensus.cc:3037] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:38.138357 15355 raft_consensus.cc:504] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35617 } } peers { permanent_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 32841 } } peers { permanent_uuid: "65bf4208c7e74528a9d469dddd42cba3" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 38793 } }
I20260806 16:06:38.139535 15066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8240f43f57456ea9259ccab27654ee" candidate_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65bf4208c7e74528a9d469dddd42cba3"
I20260806 16:06:38.139616 15256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c8240f43f57456ea9259ccab27654ee" candidate_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76beacd885e4f73a0877c49b6a6b5bb"
I20260806 16:06:38.139734 15066 raft_consensus.cc:3037] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:38.139854 15256 raft_consensus.cc:3037] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:38.141340 15066 raft_consensus.cc:2454] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f3f3486bbb1406a8bcd1378bc5ce700 in term 1.
I20260806 16:06:38.141618 15355 leader_election.cc:288] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [CANDIDATE]: Term 1 election: Requested vote from peers f76beacd885e4f73a0877c49b6a6b5bb (127.8.49.3:35617), 65bf4208c7e74528a9d469dddd42cba3 (127.8.49.2:38793)
I20260806 16:06:38.141609 15256 raft_consensus.cc:2454] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f3f3486bbb1406a8bcd1378bc5ce700 in term 1.
I20260806 16:06:38.141853 14783 leader_election.cc:302] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f3f3486bbb1406a8bcd1378bc5ce700, 65bf4208c7e74528a9d469dddd42cba3; no voters:
I20260806 16:06:38.142309 15355 raft_consensus.cc:2781] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:38.142638 15355 raft_consensus.cc:686] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [term 1 LEADER]: Becoming Leader. State: Replica: 1f3f3486bbb1406a8bcd1378bc5ce700, State: Running, Role: LEADER
I20260806 16:06:38.143096 15355 consensus_queue.cc:237] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35617 } } peers { permanent_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 32841 } } peers { permanent_uuid: "65bf4208c7e74528a9d469dddd42cba3" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 38793 } }
I20260806 16:06:38.147286 14675 catalog_manager.cc:5495] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f3f3486bbb1406a8bcd1378bc5ce700 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f3f3486bbb1406a8bcd1378bc5ce700" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 32841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65bf4208c7e74528a9d469dddd42cba3" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 38793 } health_report { overall_health: UNKNOWN } } }
I20260806 16:06:38.188162 15341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.186003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42318 (local address 127.8.49.1:32841)
0806 16:06:38.186187 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:38.186191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:38.186212 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:06:38.186431 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:38.186434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:38.186529 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:38.186626 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:38.187639 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:38.187643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:38.187645 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:38.187657 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:38.187682 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:38.187691 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:38.187824 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:38.187888 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 16:06:38.187935 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 16:06:38.194510 15140 tablet.cc:2369] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:38.196892 15066 raft_consensus.cc:1261] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3 [term 1 FOLLOWER]: Refusing update from remote peer 1f3f3486bbb1406a8bcd1378bc5ce700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:38.197021 15256 raft_consensus.cc:1261] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb [term 1 FOLLOWER]: Refusing update from remote peer 1f3f3486bbb1406a8bcd1378bc5ce700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:38.197912 15355 consensus_queue.cc:1035] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65bf4208c7e74528a9d469dddd42cba3" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 38793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:38.198346 15361 consensus_queue.cc:1035] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f76beacd885e4f73a0877c49b6a6b5bb" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 16:06:38.199011 15324 tablet.cc:2369] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:06:38.204288 14945 tablet.cc:2369] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:38.213896 15366 mvcc.cc:205] Tried to move back new op lower bound from 7315588703002898432 to 7315588702797094912. Current Snapshot: MvccSnapshot[applied={T|T < 7315588703002898432}]
I20260806 16:06:38.214265 15367 mvcc.cc:205] Tried to move back new op lower bound from 7315588703002898432 to 7315588702797094912. Current Snapshot: MvccSnapshot[applied={T|T < 7315588703002898432}]
I20260806 16:06:38.225026 14675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:38.225327 14675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:38.230149 14675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e11f162fd68b42149b4a43cec82d294a: Sending DeleteTablet for 3 replicas of tablet 4c8240f43f57456ea9259ccab27654ee
I20260806 16:06:38.231731 15227 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8240f43f57456ea9259ccab27654ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:38 UTC) from {username='slave'} at 127.0.0.1:39056
I20260806 16:06:38.232225 15038 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8240f43f57456ea9259ccab27654ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:38 UTC) from {username='slave'} at 127.0.0.1:45414
I20260806 16:06:38.232419 14857 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c8240f43f57456ea9259ccab27654ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:38 UTC) from {username='slave'} at 127.0.0.1:42306
I20260806 16:06:38.233117 15379 tablet_replica.cc:330] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: stopping tablet replica
I20260806 16:06:38.233168 15378 tablet_replica.cc:330] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: stopping tablet replica
I20260806 16:06:38.233629 15378 raft_consensus.cc:2229] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:38.233755 15379 raft_consensus.cc:2229] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:38.234069 15378 raft_consensus.cc:2258] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:38.234498 15379 raft_consensus.cc:2258] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:38.235159 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 14762
I20260806 16:06:38.235097 15380 tablet_replica.cc:330] T 4c8240f43f57456ea9259ccab27654ee P 1f3f3486bbb1406a8bcd1378bc5ce700: stopping tablet replica
I20260806 16:06:38.242277 15379 ts_tablet_manager.cc:1905] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:38.246495 15379 ts_tablet_manager.cc:1918] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:38.246604 15379 log.cc:1196] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/4c8240f43f57456ea9259ccab27654ee
I20260806 16:06:38.247000 15379 ts_tablet_manager.cc:1939] T 4c8240f43f57456ea9259ccab27654ee P 65bf4208c7e74528a9d469dddd42cba3: Deleting consensus metadata
I20260806 16:06:38.248605 14663 catalog_manager.cc:4841] TS 65bf4208c7e74528a9d469dddd42cba3 (127.8.49.2:38793): tablet 4c8240f43f57456ea9259ccab27654ee (table test-table [id=3f773e09ffdd42488d2506a47ed3a9fc]) successfully deleted
I20260806 16:06:38.248692 15378 ts_tablet_manager.cc:1905] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:38.251761 15378 ts_tablet_manager.cc:1918] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:38.251931 15378 log.cc:1196] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/4c8240f43f57456ea9259ccab27654ee
I20260806 16:06:38.252323 15378 ts_tablet_manager.cc:1939] T 4c8240f43f57456ea9259ccab27654ee P f76beacd885e4f73a0877c49b6a6b5bb: Deleting consensus metadata
I20260806 16:06:38.254016 14661 catalog_manager.cc:4841] TS f76beacd885e4f73a0877c49b6a6b5bb (127.8.49.3:35617): tablet 4c8240f43f57456ea9259ccab27654ee (table test-table [id=3f773e09ffdd42488d2506a47ed3a9fc]) successfully deleted
W20260806 16:06:38.256106 14662 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:32841 (error 108)
I20260806 16:06:38.256350 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 14948
I20260806 16:06:38.263914 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.261229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:32841 (local address 127.0.0.1:42322)
0806 16:06:38.263621 (+ 2392us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:38.263792 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:32841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1560}
W20260806 16:06:38.264189 14662 catalog_manager.cc:4584] TS 1f3f3486bbb1406a8bcd1378bc5ce700 (127.8.49.1:32841): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c8240f43f57456ea9259ccab27654ee: Network error: Client connection negotiation failed: client connection to 127.8.49.1:32841: connect: Connection refused (error 111)
I20260806 16:06:38.271613 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 15143
I20260806 16:06:38.290231 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 14637
2026-08-06T16:06:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1385 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 16:06:38.325857 8388 mini_oidc.cc:152] Starting JWKS server
I20260806 16:06:38.327821 8388 webserver.cc:460] Webserver started at https://127.0.0.1:37989/ using document root <none> and password file <none>
I20260806 16:06:38.327991 8388 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 16:06:38.328248 8388 webserver.cc:460] Webserver started at http://127.0.0.1:39821/ using document root <none> and password file <none>
2026-08-06T16:06:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:38Z Disabled control of system clock
I20260806 16:06:38.343713 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42345
--webserver_interface=127.8.49.62
--webserver_port=0
--builtin_ntp_servers=127.8.49.20:33843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:42345
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37989/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/testchainca.cert with env {}
W20260806 16:06:38.478449 15391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:38.478777 15391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:38.478852 15391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:38.483029 15391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:38.483096 15391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:38.483112 15391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:38.483130 15391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:38.483146 15391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 16:06:38.483163 15391 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 16:06:38.483177 15391 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37989/jwks.json
I20260806 16:06:38.488796 15391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:33843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:42345
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37989/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:38.490048 15391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:38.491178 15391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:38.501499 15402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:38.501636 15397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:38.501526 15398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:38.501783 15391 server_base.cc:1034] running on GCE node
I20260806 16:06:38.502919 15391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:38.504262 15391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:38.505600 15391 hybrid_clock.cc:648] HybridClock initialized: now 1786032398505587 us; error 53 us; skew 500 ppm
I20260806 16:06:38.508337 15391 webserver.cc:460] Webserver started at http://127.8.49.62:33997/ using document root <none> and password file <none>
I20260806 16:06:38.509019 15391 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:38.509092 15391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:38.509311 15391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:38.511713 15391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "875989937f4c49d5b478414e2dc7ce88"
format_stamp: "Formatted at 2026-08-06 16:06:38 on dist-test-slave-7z32"
server_key: "4c21d7a6045bea0595bb43ead9eed3c2"
server_key_iv: "ede9a4976c0a64c8efdf138161801256"
server_key_version: "encryptionkey@0"
I20260806 16:06:38.512382 15391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "875989937f4c49d5b478414e2dc7ce88"
format_stamp: "Formatted at 2026-08-06 16:06:38 on dist-test-slave-7z32"
server_key: "4c21d7a6045bea0595bb43ead9eed3c2"
server_key_iv: "ede9a4976c0a64c8efdf138161801256"
server_key_version: "encryptionkey@0"
I20260806 16:06:38.517233 15391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:06:38.520915 15411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:38.523247 15391 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:38.523434 15391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "875989937f4c49d5b478414e2dc7ce88"
format_stamp: "Formatted at 2026-08-06 16:06:38 on dist-test-slave-7z32"
server_key: "4c21d7a6045bea0595bb43ead9eed3c2"
server_key_iv: "ede9a4976c0a64c8efdf138161801256"
server_key_version: "encryptionkey@0"
I20260806 16:06:38.523603 15391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:38.545924 15391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:38.546883 15391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:38.547113 15391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:38.556156 15391 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:42345
I20260806 16:06:38.556371 15482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:42345 every 8 connection(s)
I20260806 16:06:38.557361 15391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:38.562268 15486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:38.563282 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 15391
I20260806 16:06:38.563418 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:38.563750 8388 external_mini_cluster.cc:1428] Setting key 660bfd8c2e71c02fbf9169c0f3c4f9e8
I20260806 16:06:38.572607 15486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88: Bootstrap starting.
I20260806 16:06:38.576447 15486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:38.576584 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.565699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.8.49.62:42345)
0806 16:06:38.566464 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:38.566481 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:38.566527 (+ 46us) server_negotiation.cc:407] Connection header received
0806 16:06:38.567517 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:38.567542 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:38.568020 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:38.568353 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:38.569087 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:38.570197 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:38.571786 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:38.572529 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:38.572713 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:38.574070 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:38.574076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:38.574093 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:38.574591 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:38.574660 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:38.574683 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:38.574783 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:38.575092 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 16:06:38.575556 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":198,"threads_started":1}
I20260806 16:06:38.577603 15486 log.cc:824] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:38.581347 15486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88: No bootstrap required, opened a new log
I20260806 16:06:38.581421 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:06:38.587524 15486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875989937f4c49d5b478414e2dc7ce88" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42345 } }
I20260806 16:06:38.587965 15486 raft_consensus.cc:374] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:38.588040 15486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875989937f4c49d5b478414e2dc7ce88, State: Initialized, Role: FOLLOWER
I20260806 16:06:38.588618 15486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875989937f4c49d5b478414e2dc7ce88" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42345 } }
I20260806 16:06:38.588838 15486 raft_consensus.cc:388] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:38.588944 15486 raft_consensus.cc:482] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:38.589071 15486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:38.590605 15486 raft_consensus.cc:504] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875989937f4c49d5b478414e2dc7ce88" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42345 } }
I20260806 16:06:38.591207 15486 leader_election.cc:302] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 875989937f4c49d5b478414e2dc7ce88; no voters:
I20260806 16:06:38.591836 15486 leader_election.cc:288] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:38.592226 15492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:38.592608 15492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [term 1 LEADER]: Becoming Leader. State: Replica: 875989937f4c49d5b478414e2dc7ce88, State: Running, Role: LEADER
I20260806 16:06:38.593092 15492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875989937f4c49d5b478414e2dc7ce88" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42345 } }
I20260806 16:06:38.593719 15486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:38.595414 15494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "875989937f4c49d5b478414e2dc7ce88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875989937f4c49d5b478414e2dc7ce88" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42345 } } }
I20260806 16:06:38.595621 15494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:38.595958 15496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 875989937f4c49d5b478414e2dc7ce88. Latest consensus state: current_term: 1 leader_uuid: "875989937f4c49d5b478414e2dc7ce88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875989937f4c49d5b478414e2dc7ce88" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42345 } } }
I20260806 16:06:38.596233 15496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:38.596769 15509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:38.600481 15509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:38.607667 15509 catalog_manager.cc:1347] Generated new cluster ID: 9bc6c6f6f1de42508cb595328c4918f4
I20260806 16:06:38.607824 15509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:38.632704 15509 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:38.634915 15509 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:38.643023 15509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 875989937f4c49d5b478414e2dc7ce88: Generated new TSK 0
I20260806 16:06:38.644448 15509 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 16:06:38.693118 15527 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 16:06:38.693362 15527 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 16:06:38.693856 15489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:38.666148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42184 (local address 127.8.49.62:42345)
0806 16:06:38.666347 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:38.666354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:38.666373 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:06:38.666647 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:38.666651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:38.666718 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:38.666826 (+ 108us) server_negotiation.cc:226] Negotiated authn=JWT
0806 16:06:38.667155 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:38.667753 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:38.668657 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:38.668901 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:38.669014 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:38.669432 (+ 418us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 16:06:38.693226 (+ 23794us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37989/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 16:06:38.693660 (+ 434us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42184: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37989/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 16:06:38.694021 15489 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42184: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37989/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 16:06:38.696103 15526 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:06:38.665911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42345 (local address 127.0.0.1:42184)
0806 16:06:38.666283 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:38.666299 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:38.666446 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:38.666886 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:38.666890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:38.666905 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0806 16:06:38.667030 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:38.667036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:38.667876 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:38.667880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:38.668530 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:38.668536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:38.669016 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:38.669042 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:38.669148 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:38.669200 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 16:06:38.693803 (+ 24603us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37989/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 16:06:38.694410 (+ 607us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.49.62:42345: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37989/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":137,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.49.62:42345: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37989/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 16:06:38.697368 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 15391
2026-08-06T16:06:38Z chronyd exiting
I20260806 16:06:38.747288 8388 test_util.cc:175] -----------------------------------------------
I20260806 16:06:38.747416 8388 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786032364843570-8388-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (423 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 16:06:38.748708 8388 mini_oidc.cc:152] Starting JWKS server
I20260806 16:06:38.749727 8388 webserver.cc:460] Webserver started at https://127.0.0.1:41453/ using document root <none> and password file <none>
I20260806 16:06:38.749891 8388 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 16:06:38.750226 8388 webserver.cc:460] Webserver started at http://127.0.0.1:37099/ using document root <none> and password file <none>
2026-08-06T16:06:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:38Z Disabled control of system clock
I20260806 16:06:38.765956 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:34543
--webserver_interface=127.8.49.62
--webserver_port=0
--builtin_ntp_servers=127.8.49.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:34543
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41453/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/testchainca.cert with env {}
W20260806 16:06:38.931931 15535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:38.932400 15535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:38.932452 15535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:38.937830 15535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:38.937974 15535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:38.937992 15535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:38.938014 15535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:38.938032 15535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 16:06:38.938052 15535 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 16:06:38.938067 15535 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41453/jwks.json
I20260806 16:06:38.946087 15535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:34543
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:34543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41453/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:38.947985 15535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:38.949461 15535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:38.964313 15548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:38.964200 15551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:38.964567 15535 server_base.cc:1034] running on GCE node
W20260806 16:06:38.964169 15547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:38.965600 15535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:38.967913 15535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:38.969424 15535 hybrid_clock.cc:648] HybridClock initialized: now 1786032398969351 us; error 91 us; skew 500 ppm
I20260806 16:06:38.972916 15535 webserver.cc:460] Webserver started at http://127.8.49.62:46505/ using document root <none> and password file <none>
I20260806 16:06:38.973836 15535 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:38.973932 15535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:38.974452 15535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:38.979096 15535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "4943a8de8a8a4a7a9e02c1c7404131b7"
format_stamp: "Formatted at 2026-08-06 16:06:38 on dist-test-slave-7z32"
server_key: "4dcdb133ceb4cdc5eb6e55299f4dd1ec"
server_key_iv: "941074b4c503268a6c16a2765f6ad26c"
server_key_version: "encryptionkey@0"
I20260806 16:06:38.980119 15535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "4943a8de8a8a4a7a9e02c1c7404131b7"
format_stamp: "Formatted at 2026-08-06 16:06:38 on dist-test-slave-7z32"
server_key: "4dcdb133ceb4cdc5eb6e55299f4dd1ec"
server_key_iv: "941074b4c503268a6c16a2765f6ad26c"
server_key_version: "encryptionkey@0"
I20260806 16:06:38.987295 15535 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 16:06:38.992025 15558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:38.994148 15535 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:06:38.994414 15535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "4943a8de8a8a4a7a9e02c1c7404131b7"
format_stamp: "Formatted at 2026-08-06 16:06:38 on dist-test-slave-7z32"
server_key: "4dcdb133ceb4cdc5eb6e55299f4dd1ec"
server_key_iv: "941074b4c503268a6c16a2765f6ad26c"
server_key_version: "encryptionkey@0"
I20260806 16:06:38.994652 15535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:39.012450 15535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:39.013628 15535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:39.013928 15535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:39.025139 15535 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:34543
I20260806 16:06:39.025262 15629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:34543 every 8 connection(s)
I20260806 16:06:39.026717 15535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:39.028736 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 15535
I20260806 16:06:39.028877 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:39.029309 8388 external_mini_cluster.cc:1428] Setting key 67e79b19e49ee7efc1447f03b567fbc6
I20260806 16:06:39.031945 15631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:39.039862 15631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7: Bootstrap starting.
I20260806 16:06:39.043087 15631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:39.044039 15631 log.cc:824] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:39.044435 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:39.031360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56356 (local address 127.8.49.62:34543)
0806 16:06:39.032451 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:39.032476 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:39.032534 (+ 58us) server_negotiation.cc:407] Connection header received
0806 16:06:39.033603 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:39.033640 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:39.034257 (+ 617us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:39.034691 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:39.035283 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.036568 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.038043 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.038909 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.039146 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:39.041317 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:39.041325 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:39.041345 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:39.041914 (+ 569us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:39.041998 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:39.042027 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:39.042431 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:39.042558 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 16:06:39.043160 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":197,"threads_started":1}
I20260806 16:06:39.047029 15631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7: No bootstrap required, opened a new log
I20260806 16:06:39.049016 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:06:39.051473 15631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4943a8de8a8a4a7a9e02c1c7404131b7" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34543 } }
I20260806 16:06:39.051837 15631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:39.051925 15631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4943a8de8a8a4a7a9e02c1c7404131b7, State: Initialized, Role: FOLLOWER
I20260806 16:06:39.052408 15631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4943a8de8a8a4a7a9e02c1c7404131b7" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34543 } }
I20260806 16:06:39.052582 15631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:39.052675 15631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:39.052812 15631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:39.054121 15631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4943a8de8a8a4a7a9e02c1c7404131b7" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34543 } }
I20260806 16:06:39.054522 15631 leader_election.cc:302] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4943a8de8a8a4a7a9e02c1c7404131b7; no voters:
I20260806 16:06:39.054957 15631 leader_election.cc:288] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:39.055409 15639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:39.055796 15639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [term 1 LEADER]: Becoming Leader. State: Replica: 4943a8de8a8a4a7a9e02c1c7404131b7, State: Running, Role: LEADER
I20260806 16:06:39.056267 15639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4943a8de8a8a4a7a9e02c1c7404131b7" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34543 } }
I20260806 16:06:39.056584 15631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:39.059089 15640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4943a8de8a8a4a7a9e02c1c7404131b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4943a8de8a8a4a7a9e02c1c7404131b7" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34543 } } }
I20260806 16:06:39.059408 15640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:39.059543 15643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4943a8de8a8a4a7a9e02c1c7404131b7. Latest consensus state: current_term: 1 leader_uuid: "4943a8de8a8a4a7a9e02c1c7404131b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4943a8de8a8a4a7a9e02c1c7404131b7" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34543 } } }
I20260806 16:06:39.060220 15643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7 [sys.catalog]: This master's current role is: LEADER
W20260806 16:06:39.063266 15657 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:06:39.063406 15657 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:06:39.063557 15658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:39.064778 15658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:39.074358 15658 catalog_manager.cc:1347] Generated new cluster ID: 1e3cd996bd8643ff8638d666d4d84187
I20260806 16:06:39.074507 15658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:39.096493 15658 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:39.097955 15658 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:39.109436 15658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4943a8de8a8a4a7a9e02c1c7404131b7: Generated new TSK 0
I20260806 16:06:39.110531 15658 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 16:06:39.151580 15675 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 16:06:39.151736 15675 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 16:06:39.152540 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:39.131206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.8.49.62:34543)
0806 16:06:39.131405 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:39.131420 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:39.131520 (+ 100us) server_negotiation.cc:407] Connection header received
0806 16:06:39.131767 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:39.131771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:39.131850 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:39.132026 (+ 176us) server_negotiation.cc:226] Negotiated authn=JWT
0806 16:06:39.132475 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.133906 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.135356 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.135763 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.135909 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:39.136393 (+ 484us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 16:06:39.151995 (+ 15602us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41453/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 16:06:39.152364 (+ 369us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56370: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41453/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":93}
W20260806 16:06:39.152650 15637 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56370: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41453/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 16:06:39.153085 15674 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:06:39.130994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:34543 (local address 127.0.0.1:56370)
0806 16:06:39.131390 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:39.131407 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:39.131556 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:39.132099 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:39.132102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:39.132112 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0806 16:06:39.132314 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:39.132321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.134131 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.134136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:39.135163 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:39.135179 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.135934 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.135965 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:39.136087 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:39.136133 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 16:06:39.152357 (+ 16224us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41453/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 16:06:39.152687 (+ 330us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.49.62:34543: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41453/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":142,"threads_started":1}
I20260806 16:06:39.154426 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 15535
2026-08-06T16:06:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (460 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 16:06:39.209103 8388 mini_oidc.cc:152] Starting JWKS server
I20260806 16:06:39.210247 8388 webserver.cc:460] Webserver started at https://127.0.0.1:40303/ using document root <none> and password file <none>
I20260806 16:06:39.210358 8388 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 16:06:39.210721 8388 webserver.cc:460] Webserver started at http://127.0.0.1:39817/ using document root <none> and password file <none>
2026-08-06T16:06:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:39Z Disabled control of system clock
I20260806 16:06:39.228017 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:33281
--webserver_interface=127.8.49.62
--webserver_port=0
--builtin_ntp_servers=127.8.49.20:35859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:33281
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40303/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 16:06:39.381867 15682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:39.382257 15682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:39.382303 15682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:39.387287 15682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:39.387466 15682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:39.387501 15682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:39.387533 15682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:39.387558 15682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 16:06:39.387583 15682 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 16:06:39.387609 15682 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40303/jwks.json
I20260806 16:06:39.393893 15682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:35859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:33281
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:33281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40303/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:39.395373 15682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:39.397442 15682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:06:39.410868 15682 server_base.cc:1034] running on GCE node
W20260806 16:06:39.410717 15694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:39.410717 15688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:39.411089 15690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:39.412056 15682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:39.413551 15682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:39.415046 15682 hybrid_clock.cc:648] HybridClock initialized: now 1786032399414975 us; error 93 us; skew 500 ppm
I20260806 16:06:39.417742 15682 webserver.cc:460] Webserver started at http://127.8.49.62:44089/ using document root <none> and password file <none>
I20260806 16:06:39.418673 15682 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:39.418809 15682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:39.419129 15682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:39.421965 15682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "21d2abab0a76431a81be51db088f02ce"
format_stamp: "Formatted at 2026-08-06 16:06:39 on dist-test-slave-7z32"
server_key: "b2222855796e0b4aeff317f883c7d01f"
server_key_iv: "f8cf5b1ef4ff7021644d507032047aeb"
server_key_version: "encryptionkey@0"
I20260806 16:06:39.422784 15682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "21d2abab0a76431a81be51db088f02ce"
format_stamp: "Formatted at 2026-08-06 16:06:39 on dist-test-slave-7z32"
server_key: "b2222855796e0b4aeff317f883c7d01f"
server_key_iv: "f8cf5b1ef4ff7021644d507032047aeb"
server_key_version: "encryptionkey@0"
I20260806 16:06:39.428493 15682 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:06:39.432135 15703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:39.434144 15682 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:06:39.434345 15682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "21d2abab0a76431a81be51db088f02ce"
format_stamp: "Formatted at 2026-08-06 16:06:39 on dist-test-slave-7z32"
server_key: "b2222855796e0b4aeff317f883c7d01f"
server_key_iv: "f8cf5b1ef4ff7021644d507032047aeb"
server_key_version: "encryptionkey@0"
I20260806 16:06:39.434513 15682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:39.451594 15682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:39.452824 15682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:39.453109 15682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:39.467082 15682 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:33281
I20260806 16:06:39.467199 15776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:33281 every 8 connection(s)
I20260806 16:06:39.468302 15682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:39.471720 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 15682
I20260806 16:06:39.471853 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:39.472339 8388 external_mini_cluster.cc:1428] Setting key 9808027f53442160c5d93dd2a9edfa35
I20260806 16:06:39.473245 15777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:39.483259 15777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce: Bootstrap starting.
I20260806 16:06:39.484511 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:39.474245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43638 (local address 127.8.49.62:33281)
0806 16:06:39.475142 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:39.475161 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:39.475206 (+ 45us) server_negotiation.cc:407] Connection header received
0806 16:06:39.475938 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:39.475967 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:39.476465 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:39.476819 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:39.477350 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.478527 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.480026 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.480600 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.480743 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:39.481970 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:39.481978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:39.482001 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:39.482544 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:39.482625 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:39.482652 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:39.482766 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:39.483061 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 16:06:39.483492 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":241,"threads_started":1}
I20260806 16:06:39.486930 15777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:39.488582 15777 log.cc:824] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:39.489037 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:06:39.491941 15777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce: No bootstrap required, opened a new log
I20260806 16:06:39.496950 15777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d2abab0a76431a81be51db088f02ce" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33281 } }
I20260806 16:06:39.497345 15777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:39.497442 15777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21d2abab0a76431a81be51db088f02ce, State: Initialized, Role: FOLLOWER
I20260806 16:06:39.498035 15777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d2abab0a76431a81be51db088f02ce" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33281 } }
I20260806 16:06:39.498230 15777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:39.498360 15777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:39.498504 15777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:39.499923 15777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d2abab0a76431a81be51db088f02ce" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33281 } }
I20260806 16:06:39.500454 15777 leader_election.cc:302] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21d2abab0a76431a81be51db088f02ce; no voters:
I20260806 16:06:39.500936 15777 leader_election.cc:288] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:39.501346 15784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:39.501873 15784 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [term 1 LEADER]: Becoming Leader. State: Replica: 21d2abab0a76431a81be51db088f02ce, State: Running, Role: LEADER
I20260806 16:06:39.502439 15784 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d2abab0a76431a81be51db088f02ce" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33281 } }
I20260806 16:06:39.503335 15777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:39.504964 15786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21d2abab0a76431a81be51db088f02ce. Latest consensus state: current_term: 1 leader_uuid: "21d2abab0a76431a81be51db088f02ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d2abab0a76431a81be51db088f02ce" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33281 } } }
I20260806 16:06:39.505018 15785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21d2abab0a76431a81be51db088f02ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d2abab0a76431a81be51db088f02ce" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33281 } } }
I20260806 16:06:39.505214 15785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:39.505213 15786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:39.505916 15797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:39.509895 15797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:39.518667 15797 catalog_manager.cc:1347] Generated new cluster ID: 212e24a951d941ceb1091bf12edbe323
I20260806 16:06:39.518802 15797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:39.551056 15797 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:39.552471 15797 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:39.564175 15797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21d2abab0a76431a81be51db088f02ce: Generated new TSK 0
I20260806 16:06:39.565842 15797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:06:39.591344 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:39.584454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43648 (local address 127.8.49.62:33281)
0806 16:06:39.584684 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:39.584690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:39.584918 (+ 228us) server_negotiation.cc:407] Connection header received
0806 16:06:39.585271 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:39.585275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:39.585348 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:39.585518 (+ 170us) server_negotiation.cc:226] Negotiated authn=JWT
0806 16:06:39.585890 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.586545 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.587775 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.588054 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.588200 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:39.588661 (+ 461us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 16:06:39.590711 (+ 2050us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 16:06:39.591093 (+ 382us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":111}
W20260806 16:06:39.591454 15781 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 16:06:39.593474 15822 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:06:39.584229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:33281 (local address 127.0.0.1:43648)
0806 16:06:39.584773 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:39.584792 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:39.585011 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:39.585511 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:39.585515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:39.585524 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 16:06:39.585748 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:39.585756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.586732 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.586737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:39.587597 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:39.587607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.588189 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.588233 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:39.588356 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:39.588392 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 16:06:39.591046 (+ 2654us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 16:06:39.593208 (+ 2162us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.49.62:33281: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40303/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":1735,"thread_start_us":191,"threads_started":1}
I20260806 16:06:39.594493 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 15682
E20260806 16:06:39.617455 15824 webserver.cc:533] Webserver: sslize failed:
E20260806 16:06:39.617568 15824 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T16:06:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (443 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T16:06:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:39Z Disabled control of system clock
I20260806 16:06:39.668738 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:34377
--webserver_interface=127.8.49.62
--webserver_port=0
--builtin_ntp_servers=127.8.49.20:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:34377
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/testchainca.cert with env {}
W20260806 16:06:39.828850 15830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:39.829334 15830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:39.829391 15830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:39.835453 15830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:39.835623 15830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:39.835641 15830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:39.835664 15830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:39.835687 15830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 16:06:39.835714 15830 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 16:06:39.835729 15830 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 16:06:39.846132 15830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:34377
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:34377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:39.848313 15830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:39.849906 15830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:39.863243 15840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:39.863219 15843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:39.863706 15830 server_base.cc:1034] running on GCE node
W20260806 16:06:39.863219 15838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:39.864823 15830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:39.866567 15830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:39.867836 15830 hybrid_clock.cc:648] HybridClock initialized: now 1786032399867829 us; error 80 us; skew 500 ppm
I20260806 16:06:39.871040 15830 webserver.cc:460] Webserver started at http://127.8.49.62:38651/ using document root <none> and password file <none>
I20260806 16:06:39.871917 15830 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:39.872037 15830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:39.872351 15830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:39.876302 15830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "6f1d89c73d19435fb6aeba82a9804832"
format_stamp: "Formatted at 2026-08-06 16:06:39 on dist-test-slave-7z32"
server_key: "d599684a6c27fb6b121659c559816a35"
server_key_iv: "39240a439b3addd1ad7266b67657952a"
server_key_version: "encryptionkey@0"
I20260806 16:06:39.877252 15830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "6f1d89c73d19435fb6aeba82a9804832"
format_stamp: "Formatted at 2026-08-06 16:06:39 on dist-test-slave-7z32"
server_key: "d599684a6c27fb6b121659c559816a35"
server_key_iv: "39240a439b3addd1ad7266b67657952a"
server_key_version: "encryptionkey@0"
I20260806 16:06:39.884393 15830 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 16:06:39.889034 15853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:39.891445 15830 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:06:39.891752 15830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "6f1d89c73d19435fb6aeba82a9804832"
format_stamp: "Formatted at 2026-08-06 16:06:39 on dist-test-slave-7z32"
server_key: "d599684a6c27fb6b121659c559816a35"
server_key_iv: "39240a439b3addd1ad7266b67657952a"
server_key_version: "encryptionkey@0"
I20260806 16:06:39.892021 15830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:39.905665 15830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:39.907380 15830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:39.907711 15830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:39.919839 15830 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:34377
I20260806 16:06:39.919840 15928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:34377 every 8 connection(s)
I20260806 16:06:39.921342 15830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:39.921629 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 15830
I20260806 16:06:39.921787 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:39.922413 8388 external_mini_cluster.cc:1428] Setting key ffb34260460dd141383c73ef73ab401f
I20260806 16:06:39.926494 15930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:39.935534 15930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832: Bootstrap starting.
I20260806 16:06:39.937602 15933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:39.924692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39462 (local address 127.8.49.62:34377)
0806 16:06:39.925736 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:39.925761 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:39.925848 (+ 87us) server_negotiation.cc:407] Connection header received
0806 16:06:39.926910 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:39.926940 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:39.927744 (+ 804us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:39.928181 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:39.929024 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.930748 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.932432 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:39.933106 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:39.933338 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:39.934539 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:39.934545 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:39.934565 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:06:39.935290 (+ 725us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:39.935376 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:06:39.935404 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:39.935502 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:39.935891 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 16:06:39.936347 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":195,"threads_started":1}
I20260806 16:06:39.938776 15930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:39.939826 15930 log.cc:824] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:39.942111 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:06:39.943029 15930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832: No bootstrap required, opened a new log
I20260806 16:06:39.947238 15930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d89c73d19435fb6aeba82a9804832" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34377 } }
I20260806 16:06:39.947558 15930 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:39.947662 15930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f1d89c73d19435fb6aeba82a9804832, State: Initialized, Role: FOLLOWER
I20260806 16:06:39.948305 15930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d89c73d19435fb6aeba82a9804832" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34377 } }
I20260806 16:06:39.948509 15930 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:39.948668 15930 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:39.948808 15930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:39.949965 15930 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d89c73d19435fb6aeba82a9804832" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34377 } }
I20260806 16:06:39.950323 15930 leader_election.cc:302] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f1d89c73d19435fb6aeba82a9804832; no voters:
I20260806 16:06:39.950644 15930 leader_election.cc:288] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:39.950920 15936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:39.951345 15936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [term 1 LEADER]: Becoming Leader. State: Replica: 6f1d89c73d19435fb6aeba82a9804832, State: Running, Role: LEADER
I20260806 16:06:39.951835 15936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d89c73d19435fb6aeba82a9804832" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34377 } }
I20260806 16:06:39.952245 15930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:39.954421 15938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f1d89c73d19435fb6aeba82a9804832. Latest consensus state: current_term: 1 leader_uuid: "6f1d89c73d19435fb6aeba82a9804832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d89c73d19435fb6aeba82a9804832" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34377 } } }
I20260806 16:06:39.954634 15938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:39.954491 15937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f1d89c73d19435fb6aeba82a9804832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d89c73d19435fb6aeba82a9804832" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34377 } } }
I20260806 16:06:39.954880 15937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832 [sys.catalog]: This master's current role is: LEADER
W20260806 16:06:39.958997 15954 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:06:39.959147 15954 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:06:39.959244 15961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:39.960273 15961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:39.969097 15961 catalog_manager.cc:1347] Generated new cluster ID: b5ffe5b56b7d4e8ab98eeaa4cc3aa131
I20260806 16:06:39.969278 15961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:39.994699 15961 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:39.996397 15961 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:40.028410 15961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f1d89c73d19435fb6aeba82a9804832: Generated new TSK 0
I20260806 16:06:40.029484 15961 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:06:40.974648 15933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:40.054099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.8.49.62:34377)
0806 16:06:40.054274 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:40.054282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:40.054721 (+ 439us) server_negotiation.cc:407] Connection header received
0806 16:06:40.055056 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:40.055059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:40.055139 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:40.055275 (+ 136us) server_negotiation.cc:226] Negotiated authn=JWT
0806 16:06:40.055627 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:40.056290 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:40.057659 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:40.058021 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:40.058154 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:40.058685 (+ 531us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 16:06:40.974025 (+915340us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 16:06:40.974468 (+ 443us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39464: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 16:06:40.974803 15933 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39464: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 16:06:40.974983 15973 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:06:40.053993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:34377 (local address 127.0.0.1:39464)
0806 16:06:40.054545 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:40.054563 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:40.054775 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:40.055312 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:40.055315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:40.055326 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0806 16:06:40.055485 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:40.055493 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:40.056478 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:40.056482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:40.057479 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:40.057491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:40.058161 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:40.058201 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:40.058320 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:40.058378 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 16:06:40.974444 (+916066us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 16:06:40.974707 (+ 263us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.49.62:34377: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":262,"threads_started":1}
I20260806 16:06:40.976079 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 15830
2026-08-06T16:06:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1368 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (205 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (160 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:41 dist-test-slave-7z32 krb5kdc[15990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:41 dist-test-slave-7z32 krb5kdc[15990](info): set up 2 sockets
Aug 06 16:06:41 dist-test-slave-7z32 krb5kdc[15990](info): commencing operation
krb5kdc: starting...
W20260806 16:06:42.738931 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.351s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:42 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:06:42.925125 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:44471
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:44471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:43.102656 16011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:43.103173 16011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:43.103258 16011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:43.108757 16011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:43.108901 16011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:43.108938 16011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:43.108973 16011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:43.109031 16011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:43.115785 16011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:44471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:44471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:43.117549 16011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:43.119035 16011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:43.131274 16019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:43.131349 16016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:43.131349 16022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:43.131600 16011 server_base.cc:1034] running on GCE node
I20260806 16:06:43.132406 16011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:43.133927 16011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:43.135257 16011 hybrid_clock.cc:648] HybridClock initialized: now 1786032403135223 us; error 55 us; skew 500 ppm
Aug 06 16:06:43 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032403, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:43.138715 16011 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:06:43.139889 16011 webserver.cc:460] Webserver started at http://127.8.49.62:40651/ using document root <none> and password file <none>
I20260806 16:06:43.140607 16011 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:43.140707 16011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:43.140985 16011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:43.143546 16011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "54e5e5171a194c56a905ec5f05170f51"
format_stamp: "Formatted at 2026-08-06 16:06:43 on dist-test-slave-7z32"
server_key: "c4451617ce7017de3f054cf8fd45a609"
server_key_iv: "4a1230e2715a0495aa08e920dc50fac9"
server_key_version: "encryptionkey@0"
I20260806 16:06:43.144196 16011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "54e5e5171a194c56a905ec5f05170f51"
format_stamp: "Formatted at 2026-08-06 16:06:43 on dist-test-slave-7z32"
server_key: "c4451617ce7017de3f054cf8fd45a609"
server_key_iv: "4a1230e2715a0495aa08e920dc50fac9"
server_key_version: "encryptionkey@0"
I20260806 16:06:43.149313 16011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:06:43.152804 16032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:43.154946 16011 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:43.155244 16011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "54e5e5171a194c56a905ec5f05170f51"
format_stamp: "Formatted at 2026-08-06 16:06:43 on dist-test-slave-7z32"
server_key: "c4451617ce7017de3f054cf8fd45a609"
server_key_iv: "4a1230e2715a0495aa08e920dc50fac9"
server_key_version: "encryptionkey@0"
I20260806 16:06:43.155392 16011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:43.181442 16011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:43.185765 16011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:43.186039 16011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:43.195393 16011 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:44471
I20260806 16:06:43.195452 16107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:44471 every 8 connection(s)
I20260806 16:06:43.196677 16011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:43.199539 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 16011
I20260806 16:06:43.199704 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:43.200098 8388 external_mini_cluster.cc:1428] Setting key ee6f3c3de45a3df4152f66d2d76f8c23
I20260806 16:06:43.201081 16108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:43.209188 16108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51: Bootstrap starting.
Aug 06 16:06:43 dist-test-slave-7z32 krb5kdc[15990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:43.212968 16108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:43.213994 16108 log.cc:824] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:43.217358 16108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51: No bootstrap required, opened a new log
I20260806 16:06:43.218807 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:43.202001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.8.49.62:44471)
0806 16:06:43.202689 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:43.202711 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:43.202763 (+ 52us) server_negotiation.cc:407] Connection header received
0806 16:06:43.203708 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:43.203731 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:43.204184 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:43.204587 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:43.205531 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:43.206666 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:43.207676 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:43.208237 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:43.208439 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:43.212141 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:43.212155 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:43.212178 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:43.212221 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:43.214592 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:43.215609 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:43.215616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:43.215623 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:43.215778 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:43.216215 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:43.216218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:43.216219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:43.216763 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:43.216978 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:43.217388 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 16:06:43.217816 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":226,"threads_started":1}
I20260806 16:06:43.221670 16108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e5e5171a194c56a905ec5f05170f51" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44471 } }
I20260806 16:06:43.222110 16108 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:43.222186 16108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54e5e5171a194c56a905ec5f05170f51, State: Initialized, Role: FOLLOWER
I20260806 16:06:43.222775 16108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e5e5171a194c56a905ec5f05170f51" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44471 } }
I20260806 16:06:43.222995 16108 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:43.223058 16108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:43.223148 16108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:43.224819 16108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e5e5171a194c56a905ec5f05170f51" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44471 } }
I20260806 16:06:43.225286 16108 leader_election.cc:302] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54e5e5171a194c56a905ec5f05170f51; no voters:
I20260806 16:06:43.225807 16108 leader_election.cc:288] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:43.226084 16114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:43.226478 16114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [term 1 LEADER]: Becoming Leader. State: Replica: 54e5e5171a194c56a905ec5f05170f51, State: Running, Role: LEADER
I20260806 16:06:43.226977 16114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e5e5171a194c56a905ec5f05170f51" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44471 } }
I20260806 16:06:43.227837 16108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:43.228896 16116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54e5e5171a194c56a905ec5f05170f51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e5e5171a194c56a905ec5f05170f51" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44471 } } }
I20260806 16:06:43.229102 16116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:43.229509 16119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54e5e5171a194c56a905ec5f05170f51. Latest consensus state: current_term: 1 leader_uuid: "54e5e5171a194c56a905ec5f05170f51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e5e5171a194c56a905ec5f05170f51" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44471 } } }
I20260806 16:06:43.229633 16119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:43.229759 16130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:43.234763 16130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:43.244181 16130 catalog_manager.cc:1347] Generated new cluster ID: acc5ac81b5a645f19531d86d933942dd
I20260806 16:06:43.244299 16130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:43.267998 16130 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:43.269032 16130 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:43.273416 16130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54e5e5171a194c56a905ec5f05170f51: Generated new TSK 0
I20260806 16:06:43.274264 16130 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:06:43.327396 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44471
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:43.471300 16151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:43.471676 16151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:43.471732 16151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:43.476725 16151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:43.476817 16151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:43.476890 16151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:43.484088 16151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:43.485401 16151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:43.486690 16151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:43.499593 16161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:43.499673 16159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:43.499684 16158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:43.500005 16151 server_base.cc:1034] running on GCE node
I20260806 16:06:43.500582 16151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:43.501430 16151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:43.502714 16151 hybrid_clock.cc:648] HybridClock initialized: now 1786032403502708 us; error 51 us; skew 500 ppm
Aug 06 16:06:43 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032403, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:43.506803 16151 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:06:43.508330 16151 webserver.cc:460] Webserver started at http://127.8.49.1:35993/ using document root <none> and password file <none>
I20260806 16:06:43.509159 16151 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:43.509248 16151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:43.509554 16151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:43.512507 16151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "8c7dbfa443fd4658a0facefc59f62b39"
format_stamp: "Formatted at 2026-08-06 16:06:43 on dist-test-slave-7z32"
server_key: "851de5dabd5c60736b04dfb0f9ab8db9"
server_key_iv: "72e755f3b9e44ffb288f0f1743100e8a"
server_key_version: "encryptionkey@0"
I20260806 16:06:43.513305 16151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "8c7dbfa443fd4658a0facefc59f62b39"
format_stamp: "Formatted at 2026-08-06 16:06:43 on dist-test-slave-7z32"
server_key: "851de5dabd5c60736b04dfb0f9ab8db9"
server_key_iv: "72e755f3b9e44ffb288f0f1743100e8a"
server_key_version: "encryptionkey@0"
I20260806 16:06:43.519677 16151 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:06:43.523869 16168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:43.525826 16151 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:43.526042 16151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "8c7dbfa443fd4658a0facefc59f62b39"
format_stamp: "Formatted at 2026-08-06 16:06:43 on dist-test-slave-7z32"
server_key: "851de5dabd5c60736b04dfb0f9ab8db9"
server_key_iv: "72e755f3b9e44ffb288f0f1743100e8a"
server_key_version: "encryptionkey@0"
I20260806 16:06:43.526285 16151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:43.560787 16151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:43.566303 16151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:43.566632 16151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:43.567682 16151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:43.569232 16151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:43.569336 16151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:43.569423 16151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:43.569471 16151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:43.588310 16151 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:35459
I20260806 16:06:43.588315 16348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:35459 every 8 connection(s)
I20260806 16:06:43.589679 16151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:43.599315 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 16151
I20260806 16:06:43.599524 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:43.599906 8388 external_mini_cluster.cc:1428] Setting key af37cff097764a59412ef59ad381a793
Aug 06 16:06:43 dist-test-slave-7z32 krb5kdc[15990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032403, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:43.611110 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:43.592686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:53533 (local address 127.8.49.62:44471)
0806 16:06:43.592944 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:43.592948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:43.594570 (+ 1622us) server_negotiation.cc:407] Connection header received
0806 16:06:43.596694 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:43.596698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:43.596774 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:43.596893 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:43.598374 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:43.599183 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:43.600855 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:43.601257 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:43.601484 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:43.605167 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:43.605175 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:43.605181 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:43.605226 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:43.607528 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:43.608484 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:43.608488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:43.608490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:43.608623 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:43.609229 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:43.609234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:43.609235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:43.609506 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:43.609705 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:43.610588 (+ 883us) server_negotiation.cc:299] Negotiation successful
0806 16:06:43.610902 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":84}
I20260806 16:06:43.612442 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:43.593419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44471 (local address 127.8.49.1:53533)
0806 16:06:43.594316 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:43.594385 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:43.596371 (+ 1986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:43.597124 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:43.597132 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:43.597595 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:43.598175 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:43.598190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:43.599372 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:43.599375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:43.600594 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:43.600609 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:43.601564 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:43.601571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:43.601813 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:43.603033 (+ 1220us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:43.603056 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:43.604885 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:43.607754 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:43.607766 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:43.607793 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:43.608296 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:43.608774 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:43.608780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:43.608785 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:43.609024 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:43.609723 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:43.609734 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:43.610179 (+ 445us) client_negotiation.cc:769] Sending connection context
0806 16:06:43.610453 (+ 274us) client_negotiation.cc:240] Negotiation successful
0806 16:06:43.611027 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":233,"threads_started":1}
I20260806 16:06:43.614362 16349 heartbeater.cc:347] Connected to a master server at 127.8.49.62:44471
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
I20260806 16:06:43.614876 16349 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:43.615717 16349 heartbeater.cc:511] Master 127.8.49.62:44471 requested a full tablet report, sending...
I20260806 16:06:43.618603 16053 ts_manager.cc:194] Registered new tserver with Master: 8c7dbfa443fd4658a0facefc59f62b39 (127.8.49.1:35459)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:06:43.620771 16053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:53533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:06:43.647481 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44471
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:43.788329 16356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:43.788707 16356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:43.788785 16356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:43.793236 16356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:43.793365 16356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:43.793465 16356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:43.799540 16356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:43.800894 16356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:43.802084 16356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:43.813710 16364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:43.813963 16366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:43.813980 16370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:43.814291 16356 server_base.cc:1034] running on GCE node
I20260806 16:06:43.814971 16356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:43.815893 16356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:43.817191 16356 hybrid_clock.cc:648] HybridClock initialized: now 1786032403817165 us; error 74 us; skew 500 ppm
Aug 06 16:06:43 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032403, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:43.821008 16356 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:06:43.822257 16356 webserver.cc:460] Webserver started at http://127.8.49.2:39393/ using document root <none> and password file <none>
I20260806 16:06:43.823031 16356 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:43.823141 16356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:43.823418 16356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:43.825780 16356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "862bd8d6377f471fbd3e183a06d722dd"
format_stamp: "Formatted at 2026-08-06 16:06:43 on dist-test-slave-7z32"
server_key: "1c499dbb52f7ae05a4779341eed747ce"
server_key_iv: "579d5989a2526ba5a524c05fa9450871"
server_key_version: "encryptionkey@0"
I20260806 16:06:43.826418 16356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "862bd8d6377f471fbd3e183a06d722dd"
format_stamp: "Formatted at 2026-08-06 16:06:43 on dist-test-slave-7z32"
server_key: "1c499dbb52f7ae05a4779341eed747ce"
server_key_iv: "579d5989a2526ba5a524c05fa9450871"
server_key_version: "encryptionkey@0"
I20260806 16:06:43.831777 16356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:06:43.835623 16381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:43.837550 16356 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:43.837752 16356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "862bd8d6377f471fbd3e183a06d722dd"
format_stamp: "Formatted at 2026-08-06 16:06:43 on dist-test-slave-7z32"
server_key: "1c499dbb52f7ae05a4779341eed747ce"
server_key_iv: "579d5989a2526ba5a524c05fa9450871"
server_key_version: "encryptionkey@0"
I20260806 16:06:43.837910 16356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:43.863718 16356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:43.867487 16356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:43.867754 16356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:43.868476 16356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:43.869486 16356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:43.869571 16356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:43.869648 16356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:43.869694 16356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:43.886233 16356 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:43847
I20260806 16:06:43.886353 16549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:43847 every 8 connection(s)
I20260806 16:06:43.888119 16356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:43.890987 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 16356
I20260806 16:06:43.891149 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:43.891773 8388 external_mini_cluster.cc:1428] Setting key 3663b79178dd842f8e5db96bc4fd6de4
Aug 06 16:06:43 dist-test-slave-7z32 krb5kdc[15990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032403, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:06:43.912447 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:43.892024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:60311 (local address 127.8.49.62:44471)
0806 16:06:43.892295 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:43.892301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:43.893956 (+ 1655us) server_negotiation.cc:407] Connection header received
0806 16:06:43.896421 (+ 2465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:43.896425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:43.896509 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:43.896641 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:43.898814 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:43.899464 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:43.901159 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:43.901412 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:43.901535 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:43.906898 (+ 5363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:43.906902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:43.906905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:43.906941 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:43.909505 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:43.910282 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:43.910287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:43.910289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:43.910391 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:43.910910 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:43.910915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:43.910918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:43.911252 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:43.911384 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:43.912052 (+ 668us) server_negotiation.cc:299] Negotiation successful
0806 16:06:43.912245 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:06:43.913797 16555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:43.892827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44471 (local address 127.8.49.2:60311)
0806 16:06:43.893745 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:43.893821 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:43.896056 (+ 2235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:43.896889 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:43.896899 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:43.897615 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:43.898306 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:43.898326 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:43.899781 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:43.899785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:43.900989 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:43.901004 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:43.901617 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:43.901623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:43.901820 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:43.903375 (+ 1555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:43.903397 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:43.906559 (+ 3162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:43.909727 (+ 3168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:43.909736 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:43.909757 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:43.910138 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:43.910510 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:43.910514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:43.910516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:43.910719 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:43.911386 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:43.911398 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:43.911745 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 16:06:43.911982 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 16:06:43.912553 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":193,"threads_started":1}
I20260806 16:06:43.915416 16551 heartbeater.cc:347] Connected to a master server at 127.8.49.62:44471
I20260806 16:06:43.915771 16551 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:43.916453 16551 heartbeater.cc:511] Master 127.8.49.62:44471 requested a full tablet report, sending...
I20260806 16:06:43.918081 16053 ts_manager.cc:194] Registered new tserver with Master: 862bd8d6377f471fbd3e183a06d722dd (127.8.49.2:43847)
I20260806 16:06:43.918768 16053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:60311
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:06:43.944384 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44471
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:44.128968 16558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:44.129472 16558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:44.129565 16558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:44.135089 16558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:44.135196 16558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:44.135309 16558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:44.143875 16558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:44.145714 16558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:44.147482 16558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:44.161620 16572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:44.161613 16579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:44.161710 16570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:44.162348 16558 server_base.cc:1034] running on GCE node
I20260806 16:06:44.163149 16558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:44.164170 16558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:44.165508 16558 hybrid_clock.cc:648] HybridClock initialized: now 1786032404165438 us; error 51 us; skew 500 ppm
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:44.171128 16558 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:06:44.172852 16558 webserver.cc:460] Webserver started at http://127.8.49.3:35173/ using document root <none> and password file <none>
I20260806 16:06:44.173794 16558 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:44.174063 16558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:44.174445 16558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:44.178103 16558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "9e0eafb180a44cc8b273391873cd8bed"
format_stamp: "Formatted at 2026-08-06 16:06:44 on dist-test-slave-7z32"
server_key: "f0cb8f2d9a1d5a4ecb5a403ac96f8e35"
server_key_iv: "aa07efa6133da615526b6cc296c6e121"
server_key_version: "encryptionkey@0"
I20260806 16:06:44.179141 16558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "9e0eafb180a44cc8b273391873cd8bed"
format_stamp: "Formatted at 2026-08-06 16:06:44 on dist-test-slave-7z32"
server_key: "f0cb8f2d9a1d5a4ecb5a403ac96f8e35"
server_key_iv: "aa07efa6133da615526b6cc296c6e121"
server_key_version: "encryptionkey@0"
I20260806 16:06:44.187572 16558 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.004s
I20260806 16:06:44.192098 16589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:44.194006 16558 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:44.194212 16558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "9e0eafb180a44cc8b273391873cd8bed"
format_stamp: "Formatted at 2026-08-06 16:06:44 on dist-test-slave-7z32"
server_key: "f0cb8f2d9a1d5a4ecb5a403ac96f8e35"
server_key_iv: "aa07efa6133da615526b6cc296c6e121"
server_key_version: "encryptionkey@0"
I20260806 16:06:44.194428 16558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:44.210451 16558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:44.214653 16558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:44.214979 16558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:44.215806 16558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:44.217202 16558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:44.217312 16558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:44.217401 16558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:44.217453 16558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:44.231977 16558 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:35485
I20260806 16:06:44.232231 16769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:35485 every 8 connection(s)
I20260806 16:06:44.233382 16558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:44.239316 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 16558
I20260806 16:06:44.239508 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:44.239856 8388 external_mini_cluster.cc:1428] Setting key dae1a507b0377064e1706a10e345a41f
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:06:44.255103 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:44.236343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:60307 (local address 127.8.49.62:44471)
0806 16:06:44.236537 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:44.236542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:44.238091 (+ 1549us) server_negotiation.cc:407] Connection header received
0806 16:06:44.240466 (+ 2375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:44.240471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:44.240588 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:44.240720 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:44.242912 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:44.243884 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:44.245289 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:44.245564 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:44.245678 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:44.249109 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:44.249113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:44.249116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:44.249158 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:44.251980 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:44.252747 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:44.252751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:44.252754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:44.252872 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:44.253303 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:44.253307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:44.253309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:44.253557 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:44.253685 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:44.254479 (+ 794us) server_negotiation.cc:299] Negotiation successful
0806 16:06:44.254881 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 16:06:44.256086 16773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:44.236975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44471 (local address 127.8.49.3:60307)
0806 16:06:44.237773 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:44.237836 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:44.240053 (+ 2217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:44.241056 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:44.241066 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:44.241800 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:44.242689 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:44.242709 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:44.244055 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:44.244058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:44.245099 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:44.245112 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:44.245738 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:44.245741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:44.245911 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:44.246687 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:44.246705 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:44.248853 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:44.252172 (+ 3319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:44.252182 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:44.252221 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:44.252605 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:44.252991 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:44.252995 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:44.252997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:44.253181 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:44.253686 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:44.253696 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:44.254023 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 16:06:44.254286 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 16:06:44.254698 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":193,"threads_started":1}
I20260806 16:06:44.258080 16770 heartbeater.cc:347] Connected to a master server at 127.8.49.62:44471
I20260806 16:06:44.258406 16770 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:44.259095 16770 heartbeater.cc:511] Master 127.8.49.62:44471 requested a full tablet report, sending...
I20260806 16:06:44.260695 16053 ts_manager.cc:194] Registered new tserver with Master: 9e0eafb180a44cc8b273391873cd8bed (127.8.49.3:35485)
I20260806 16:06:44.261370 16053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:60307
I20260806 16:06:44.270179 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/test-admin.keytab.
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:44.294737 8388 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 16:06:44.298970 16782 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:06:44.297109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.8.49.1:35459)
0806 16:06:44.297581 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:44.297590 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:44.297626 (+ 36us) server_negotiation.cc:407] Connection header received
0806 16:06:44.297698 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:44.297706 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:44.297946 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:44.298117 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:44.298725 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54286: BlockingRecv error: recv got EOF from 127.0.0.1:54286 (error 108)
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":182,"threads_started":1}
W20260806 16:06:44.299343 8388 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/krb5cc' of type FILE
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:44.300886 8388 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:44.302145 8388 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:06:44.316551 16782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:44.303502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54302 (local address 127.8.49.1:35459)
0806 16:06:44.303644 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:44.303647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:44.303688 (+ 41us) server_negotiation.cc:407] Connection header received
0806 16:06:44.303898 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:44.303900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:44.303950 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:44.304049 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:44.304763 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:44.305623 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:44.306696 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:44.307317 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:44.307512 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:44.311212 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:44.311227 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:44.311247 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:44.311284 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:44.313719 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:44.314552 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:44.314560 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:44.314561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:44.314677 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:44.315355 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:44.315358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:44.315359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:44.315697 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:44.315970 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:44.316234 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 16:06:44.316415 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 16:06:44.323403 16782 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:06:44.321712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54310 (local address 127.8.49.1:35459)
0806 16:06:44.321896 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:44.321901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:44.322093 (+ 192us) server_negotiation.cc:407] Connection header received
0806 16:06:44.322215 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:44.322218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:44.322268 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:44.322359 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:44.323250 (+ 891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54310: BlockingRecv error: recv got EOF from 127.0.0.1:54310 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 16:06:44.323869 8388 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/krb5cc' of type FILE
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:44.325464 8388 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:44.326807 8388 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:06:44.340600 16782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:44.328391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54316 (local address 127.8.49.1:35459)
0806 16:06:44.328803 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:44.328809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:44.328984 (+ 175us) server_negotiation.cc:407] Connection header received
0806 16:06:44.329112 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:44.329115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:44.329215 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:44.329311 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:44.330150 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:44.330958 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:44.332112 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:44.332470 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:44.332606 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:44.336197 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:44.336200 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:44.336202 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:44.336239 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:44.338030 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:44.338758 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:44.338762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:44.338763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:44.339013 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:44.339453 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:44.339457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:44.339459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:44.339698 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:44.339912 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:44.340198 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 16:06:44.340346 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248}
W20260806 16:06:44.345767 16782 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:06:44.343847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54322 (local address 127.8.49.1:35459)
0806 16:06:44.344032 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:44.344038 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:44.344220 (+ 182us) server_negotiation.cc:407] Connection header received
0806 16:06:44.344401 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:44.344404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:44.344479 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:44.344574 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:44.345652 (+ 1078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54322: BlockingRecv error: recv got EOF from 127.0.0.1:54322 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 16:06:44.346350 8388 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786032364843570-8388-0/krb5kdc/krb5cc' of type FILE
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:44.348014 8388 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:44.349256 8388 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[15990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:06:44.364187 16782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:44.351169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54328 (local address 127.8.49.1:35459)
0806 16:06:44.351420 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:44.351427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:44.351797 (+ 370us) server_negotiation.cc:407] Connection header received
0806 16:06:44.352037 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:44.352040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:44.352111 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:44.352217 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:44.353087 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:44.353707 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:44.355180 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:44.355524 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:44.355641 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:44.359147 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:44.359154 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:44.359157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:44.359208 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:44.361434 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:44.362193 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:44.362207 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:44.362209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:44.362323 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:44.362681 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:44.362684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:44.362685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:44.363064 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:44.363223 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:44.363727 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 16:06:44.363905 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 16:06:44.366340 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 16151
I20260806 16:06:44.387315 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 16356
I20260806 16:06:44.418670 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 16558
I20260806 16:06:44.437506 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 16011
2026-08-06T16:06:44Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3087 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[16805](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[16805](info): set up 2 sockets
Aug 06 16:06:44 dist-test-slave-7z32 krb5kdc[16805](info): commencing operation
krb5kdc: starting...
W20260806 16:06:45.837654 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.361s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:45 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.62.keytab.
Entry for principal oryx/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.62.keytab.
I20260806 16:06:46.017895 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:45969
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.62.keytab
--principal=oryx/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:45969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:46.148599 16823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:46.148931 16823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:46.148972 16823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:46.153195 16823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:46.153283 16823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:46.153306 16823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:46.153323 16823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:46.153339 16823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:46.159284 16823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:45969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.62.keytab
--principal=oryx/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:45969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:46.160538 16823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:46.161636 16823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:46.171790 16833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:46.171838 16829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:46.171994 16830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:46.172600 16823 server_base.cc:1034] running on GCE node
I20260806 16:06:46.173594 16823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:46.175361 16823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:46.177484 16823 hybrid_clock.cc:648] HybridClock initialized: now 1786032406177286 us; error 172 us; skew 500 ppm
Aug 06 16:06:46 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032406, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:46.182487 16823 init.cc:377] Logged in from keytab as oryx/127.8.49.62@KRBTEST.COM (short username oryx)
I20260806 16:06:46.184091 16823 webserver.cc:460] Webserver started at http://127.8.49.62:35789/ using document root <none> and password file <none>
I20260806 16:06:46.184960 16823 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:46.185082 16823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:46.185411 16823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:46.189204 16823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "62f1112cc8e447e19ce70e0eab5a186b"
format_stamp: "Formatted at 2026-08-06 16:06:46 on dist-test-slave-7z32"
server_key: "c7ed54afe045c2c15edb8485f0301210"
server_key_iv: "5ab501105ac9f145f727677d6c095669"
server_key_version: "encryptionkey@0"
I20260806 16:06:46.189998 16823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "62f1112cc8e447e19ce70e0eab5a186b"
format_stamp: "Formatted at 2026-08-06 16:06:46 on dist-test-slave-7z32"
server_key: "c7ed54afe045c2c15edb8485f0301210"
server_key_iv: "5ab501105ac9f145f727677d6c095669"
server_key_version: "encryptionkey@0"
I20260806 16:06:46.196060 16823 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 16:06:46.200290 16842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:46.202627 16823 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260806 16:06:46.202971 16823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "62f1112cc8e447e19ce70e0eab5a186b"
format_stamp: "Formatted at 2026-08-06 16:06:46 on dist-test-slave-7z32"
server_key: "c7ed54afe045c2c15edb8485f0301210"
server_key_iv: "5ab501105ac9f145f727677d6c095669"
server_key_version: "encryptionkey@0"
I20260806 16:06:46.203214 16823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:46.220593 16823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:46.224998 16823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:46.225260 16823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:46.235576 16823 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:45969
I20260806 16:06:46.236033 16920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:45969 every 8 connection(s)
I20260806 16:06:46.237241 16823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:46.239972 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 16823
I20260806 16:06:46.240177 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:46.240540 8388 external_mini_cluster.cc:1428] Setting key edc77e85ca6fe8eb74f1aeafda1a383a
I20260806 16:06:46.241530 16921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:46.250202 16921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b: Bootstrap starting.
Aug 06 16:06:46 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
I20260806 16:06:46.253528 16921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:46.254410 16921 log.cc:824] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:46.257607 16921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b: No bootstrap required, opened a new log
I20260806 16:06:46.263270 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:46.242499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52164 (local address 127.8.49.62:45969)
0806 16:06:46.243139 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:46.243159 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:46.243211 (+ 52us) server_negotiation.cc:407] Connection header received
0806 16:06:46.244064 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:46.244087 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:46.244558 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:46.245039 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:46.246347 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:46.247714 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:46.249277 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:46.250056 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:46.250243 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:46.254070 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:46.254078 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:46.254104 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:46.254155 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:46.259193 (+ 5038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:46.260054 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:46.260063 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:46.260071 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:46.260233 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:46.260630 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:46.260633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:46.260634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:46.261198 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:46.261381 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:46.261802 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 16:06:46.262170 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":218,"threads_started":1}
I20260806 16:06:46.263865 16921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } }
I20260806 16:06:46.264267 16921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:46.264328 16921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62f1112cc8e447e19ce70e0eab5a186b, State: Initialized, Role: FOLLOWER
I20260806 16:06:46.264940 16921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } }
I20260806 16:06:46.265105 16921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:46.265206 16921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:46.265327 16921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:46.267200 16921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } }
I20260806 16:06:46.267711 16921 leader_election.cc:302] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62f1112cc8e447e19ce70e0eab5a186b; no voters:
I20260806 16:06:46.268236 16921 leader_election.cc:288] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:46.268442 16929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:46.268795 16929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 1 LEADER]: Becoming Leader. State: Replica: 62f1112cc8e447e19ce70e0eab5a186b, State: Running, Role: LEADER
I20260806 16:06:46.269659 16929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } }
I20260806 16:06:46.270756 16921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:46.272497 16934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62f1112cc8e447e19ce70e0eab5a186b. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } }
I20260806 16:06:46.272729 16934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:46.272455 16932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } }
I20260806 16:06:46.272956 16932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:46.273304 16948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:46.276834 16948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:46.286473 16948 catalog_manager.cc:1347] Generated new cluster ID: 0b8b98af43d743aa824a44346403a0cc
I20260806 16:06:46.286614 16948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:46.294430 16948 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:46.296208 16948 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:46.304140 16948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b: Generated new TSK 0
I20260806 16:06:46.305497 16948 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab.
Entry for principal oryx/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab.
I20260806 16:06:46.361271 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45969
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab
--principal=oryx/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:46.532696 16964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:46.533123 16964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:46.533180 16964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:46.537798 16964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:46.537901 16964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:46.537973 16964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:46.544338 16964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab
--principal=oryx/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:46.545882 16964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:46.547231 16964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:46.561089 16970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:46.561110 16971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:46.561110 16975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:46.561170 16964 server_base.cc:1034] running on GCE node
I20260806 16:06:46.562036 16964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:46.563097 16964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:46.564402 16964 hybrid_clock.cc:648] HybridClock initialized: now 1786032406564373 us; error 50 us; skew 500 ppm
Aug 06 16:06:46 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032406, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:46.568401 16964 init.cc:377] Logged in from keytab as oryx/127.8.49.1@KRBTEST.COM (short username oryx)
I20260806 16:06:46.569744 16964 webserver.cc:460] Webserver started at http://127.8.49.1:39373/ using document root <none> and password file <none>
I20260806 16:06:46.570885 16964 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:46.571024 16964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:46.571328 16964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:46.574501 16964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "702e742c23214b40aa48256127f733f2"
format_stamp: "Formatted at 2026-08-06 16:06:46 on dist-test-slave-7z32"
server_key: "c8970b81bb7f7f542718332847301b55"
server_key_iv: "cdcb877add40c1dcf089acc241ad8fad"
server_key_version: "encryptionkey@0"
I20260806 16:06:46.575354 16964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "702e742c23214b40aa48256127f733f2"
format_stamp: "Formatted at 2026-08-06 16:06:46 on dist-test-slave-7z32"
server_key: "c8970b81bb7f7f542718332847301b55"
server_key_iv: "cdcb877add40c1dcf089acc241ad8fad"
server_key_version: "encryptionkey@0"
I20260806 16:06:46.581472 16964 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:06:46.585387 16987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:46.587471 16964 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:46.587692 16964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "702e742c23214b40aa48256127f733f2"
format_stamp: "Formatted at 2026-08-06 16:06:46 on dist-test-slave-7z32"
server_key: "c8970b81bb7f7f542718332847301b55"
server_key_iv: "cdcb877add40c1dcf089acc241ad8fad"
server_key_version: "encryptionkey@0"
I20260806 16:06:46.587895 16964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:46.599195 16964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:46.604132 16964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:46.604442 16964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:46.605499 16964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:46.606796 16964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:46.606927 16964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 16:06:46.607008 16964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:46.607045 16964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:46.621644 16964 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:40541
I20260806 16:06:46.621703 17143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:40541 every 8 connection(s)
I20260806 16:06:46.623080 16964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:46.627118 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 16964
I20260806 16:06:46.627271 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:46.627655 8388 external_mini_cluster.cc:1428] Setting key e2bd21ab9155557e0d3219026d1a317f
Aug 06 16:06:46 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032406, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.8.49.2@KRBTEST.COM; defaulting to no policy
I20260806 16:06:46.644964 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:46.625758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:55125 (local address 127.8.49.62:45969)
0806 16:06:46.625995 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:46.626000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:46.627548 (+ 1548us) server_negotiation.cc:407] Connection header received
0806 16:06:46.628769 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:46.628772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:46.628822 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:46.628943 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:46.631024 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:46.632032 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:46.634247 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:46.634775 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:46.634974 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:46.638794 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:46.638800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:46.638803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:46.638896 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:46.641295 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:46.642423 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:46.642431 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:46.642435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:46.642567 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:46.643156 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:46.643160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:46.643162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:46.643435 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:46.643574 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:46.644669 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0806 16:06:46.644822 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.2@KRBTEST.COM" created.
I20260806 16:06:46.646737 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:46.626223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.1:55125)
0806 16:06:46.627305 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:46.627372 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:46.628505 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:46.629314 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:46.629324 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:46.629856 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:46.630723 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:46.630748 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:46.632297 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:46.632304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:46.634019 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:46.634041 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:46.635054 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:46.635058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:46.635302 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:46.636284 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:46.636306 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:46.638494 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:46.641577 (+ 3083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:46.641586 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:46.641609 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:46.642183 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:46.642709 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:46.642712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:46.642713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:46.643022 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:46.643680 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:46.643695 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:46.644296 (+ 601us) client_negotiation.cc:769] Sending connection context
0806 16:06:46.644568 (+ 272us) client_negotiation.cc:240] Negotiation successful
0806 16:06:46.645355 (+ 787us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":190,"threads_started":1}
I20260806 16:06:46.648787 17144 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45969
I20260806 16:06:46.649139 17144 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:46.649950 17144 heartbeater.cc:511] Master 127.8.49.62:45969 requested a full tablet report, sending...
I20260806 16:06:46.652683 16871 ts_manager.cc:194] Registered new tserver with Master: 702e742c23214b40aa48256127f733f2 (127.8.49.1:40541)
I20260806 16:06:46.654676 16871 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:55125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab.
Entry for principal oryx/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab.
I20260806 16:06:46.672200 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45969
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab
--principal=oryx/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:46.811441 17150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:46.811812 17150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:46.811892 17150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:46.816278 17150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:46.816392 17150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:46.816495 17150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:46.822300 17150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab
--principal=oryx/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:46.823679 17150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:46.824863 17150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:06:46.836284 17150 server_base.cc:1034] running on GCE node
W20260806 16:06:46.836261 17157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:46.836297 17161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:46.836261 17158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:46.837426 17150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:46.838299 17150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:46.839546 17150 hybrid_clock.cc:648] HybridClock initialized: now 1786032406839519 us; error 50 us; skew 500 ppm
Aug 06 16:06:46 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032406, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:46.843351 17150 init.cc:377] Logged in from keytab as oryx/127.8.49.2@KRBTEST.COM (short username oryx)
I20260806 16:06:46.844478 17150 webserver.cc:460] Webserver started at http://127.8.49.2:43681/ using document root <none> and password file <none>
I20260806 16:06:46.845155 17150 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:46.845257 17150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:46.845534 17150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:46.847918 17150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "420507bd250744c2b3a2a83baf2cb501"
format_stamp: "Formatted at 2026-08-06 16:06:46 on dist-test-slave-7z32"
server_key: "07fd207ab7353ff884f066ed953a4e06"
server_key_iv: "028f89309759f1b24bf39c7a725c497e"
server_key_version: "encryptionkey@0"
I20260806 16:06:46.848567 17150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "420507bd250744c2b3a2a83baf2cb501"
format_stamp: "Formatted at 2026-08-06 16:06:46 on dist-test-slave-7z32"
server_key: "07fd207ab7353ff884f066ed953a4e06"
server_key_iv: "028f89309759f1b24bf39c7a725c497e"
server_key_version: "encryptionkey@0"
I20260806 16:06:46.853278 17150 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:06:46.856444 17171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:46.858117 17150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 16:06:46.858319 17150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "420507bd250744c2b3a2a83baf2cb501"
format_stamp: "Formatted at 2026-08-06 16:06:46 on dist-test-slave-7z32"
server_key: "07fd207ab7353ff884f066ed953a4e06"
server_key_iv: "028f89309759f1b24bf39c7a725c497e"
server_key_version: "encryptionkey@0"
I20260806 16:06:46.858498 17150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:46.878211 17150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:46.882359 17150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:46.882633 17150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:46.883442 17150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:46.884546 17150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:46.884603 17150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:46.884646 17150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:46.884662 17150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:46.898193 17150 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:44165
I20260806 16:06:46.898320 17339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:44165 every 8 connection(s)
I20260806 16:06:46.899457 17150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:46.904547 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 17150
I20260806 16:06:46.904736 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:46.905165 8388 external_mini_cluster.cc:1428] Setting key 2dd70a509d1f15d2aeda4cc7bf10642c
Aug 06 16:06:46 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032406, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:06:46.922264 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:46.902921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:41887 (local address 127.8.49.62:45969)
0806 16:06:46.903185 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:46.903190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:46.904512 (+ 1322us) server_negotiation.cc:407] Connection header received
0806 16:06:46.906659 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:46.906663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:46.906735 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:46.906865 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:46.909710 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:46.910436 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:46.912242 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:46.912660 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:46.912855 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:46.916894 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:46.916900 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:46.916905 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:46.916962 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:46.918985 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:46.919944 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:46.919948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:46.919949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:46.920040 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:46.920833 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:46.920836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:46.920837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:46.921066 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:46.921156 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:46.921981 (+ 825us) server_negotiation.cc:299] Negotiation successful
0806 16:06:46.922126 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.3@KRBTEST.COM" created.
I20260806 16:06:46.924049 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:46.903313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.2:41887)
0806 16:06:46.904076 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:46.904170 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:46.906357 (+ 2187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:46.907167 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:46.907180 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:46.907950 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:46.909206 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:46.909457 (+ 251us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:46.910658 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:46.910665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:46.912042 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:46.912057 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:46.912940 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:46.912943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:46.913150 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:46.914014 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:46.914036 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:46.916540 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:46.919216 (+ 2676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:46.919258 (+ 42us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:46.919280 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:46.919772 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:46.920148 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:46.920150 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:46.920152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:46.920571 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:46.921207 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:46.921221 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:46.921613 (+ 392us) client_negotiation.cc:769] Sending connection context
0806 16:06:46.921884 (+ 271us) client_negotiation.cc:240] Negotiation successful
0806 16:06:46.922517 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":144,"threads_started":1}
I20260806 16:06:46.925794 17340 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45969
I20260806 16:06:46.926198 17340 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:46.927027 17340 heartbeater.cc:511] Master 127.8.49.62:45969 requested a full tablet report, sending...
I20260806 16:06:46.928576 16871 ts_manager.cc:194] Registered new tserver with Master: 420507bd250744c2b3a2a83baf2cb501 (127.8.49.2:44165)
I20260806 16:06:46.929478 16871 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:41887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab.
Entry for principal oryx/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab.
I20260806 16:06:46.947268 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45969
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab
--principal=oryx/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:47.112407 17349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:47.112891 17349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:47.112989 17349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:47.118599 17349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:47.118769 17349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:47.118917 17349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:47.126858 17349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab
--principal=oryx/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:47.128628 17349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:47.130460 17349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:47.144263 17358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:47.144248 17365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:47.144248 17359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:47.144855 17349 server_base.cc:1034] running on GCE node
I20260806 16:06:47.145715 17349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:47.146624 17349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:47.147866 17349 hybrid_clock.cc:648] HybridClock initialized: now 1786032407147837 us; error 46 us; skew 500 ppm
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:47.151957 17349 init.cc:377] Logged in from keytab as oryx/127.8.49.3@KRBTEST.COM (short username oryx)
I20260806 16:06:47.153273 17349 webserver.cc:460] Webserver started at http://127.8.49.3:38325/ using document root <none> and password file <none>
I20260806 16:06:47.154161 17349 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:47.154296 17349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:47.154585 17349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:47.157236 17349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "2c3dc20b5f304e69a6cf42f8baa83e17"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "7a2ea1c5d6700457185004629252ea85"
server_key_iv: "004651382b7ddde70aed8d9ba0011465"
server_key_version: "encryptionkey@0"
I20260806 16:06:47.157928 17349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "2c3dc20b5f304e69a6cf42f8baa83e17"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "7a2ea1c5d6700457185004629252ea85"
server_key_iv: "004651382b7ddde70aed8d9ba0011465"
server_key_version: "encryptionkey@0"
I20260806 16:06:47.163702 17349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:06:47.167119 17376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:47.168969 17349 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:47.169135 17349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "2c3dc20b5f304e69a6cf42f8baa83e17"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "7a2ea1c5d6700457185004629252ea85"
server_key_iv: "004651382b7ddde70aed8d9ba0011465"
server_key_version: "encryptionkey@0"
I20260806 16:06:47.169304 17349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:47.184846 17349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:47.189414 17349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:47.189641 17349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:47.190338 17349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:47.191547 17349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:47.191618 17349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:47.191668 17349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:47.191685 17349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:47.204015 17349 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:43059
I20260806 16:06:47.204098 17537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:43059 every 8 connection(s)
I20260806 16:06:47.205219 17349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:47.209393 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 17349
I20260806 16:06:47.209573 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:47.209975 8388 external_mini_cluster.cc:1428] Setting key 50048beffc5a2e7d327a2e48b878c0af
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
I20260806 16:06:47.226313 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.208151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:35239 (local address 127.8.49.62:45969)
0806 16:06:47.208394 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.208400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.209719 (+ 1319us) server_negotiation.cc:407] Connection header received
0806 16:06:47.211703 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.211707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.211776 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.211889 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.213679 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.214256 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.215808 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.216110 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.216241 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.220443 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.220447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.220450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.220484 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.222363 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.223239 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.223245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.223246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.223371 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.224120 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.224123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.224125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.224467 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.224597 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.225847 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.226069 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 16:06:47.227653 17540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.208751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.3:35239)
0806 16:06:47.209482 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:47.209552 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:47.211429 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:47.212144 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:47.212153 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:47.212674 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:47.213489 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.213506 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.214425 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.214428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.215647 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.215661 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.216408 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.216411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.216604 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.217485 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:47.217506 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:47.219855 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:47.222667 (+ 2812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.222675 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.222694 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.223097 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.223563 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.223566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.223567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.223831 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.224712 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:47.224726 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:47.225346 (+ 620us) client_negotiation.cc:769] Sending connection context
0806 16:06:47.225661 (+ 315us) client_negotiation.cc:240] Negotiation successful
0806 16:06:47.226211 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":253,"threads_started":1}
I20260806 16:06:47.229744 17538 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45969
I20260806 16:06:47.230111 17538 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:47.230760 17538 heartbeater.cc:511] Master 127.8.49.62:45969 requested a full tablet report, sending...
I20260806 16:06:47.232306 16871 ts_manager.cc:194] Registered new tserver with Master: 2c3dc20b5f304e69a6cf42f8baa83e17 (127.8.49.3:43059)
I20260806 16:06:47.233280 16871 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:35239
I20260806 16:06:47.239315 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.49.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.61.keytab.
Entry for principal oryx/127.8.49.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.61.keytab.
I20260806 16:06:47.280594 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.61:38683
--webserver_interface=127.8.49.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.61.keytab
--principal=oryx/127.8.49.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:45969,127.8.49.61:38683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:47.419396 17545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:47.419739 17545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:47.419783 17545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:47.424098 17545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:47.424194 17545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:47.424214 17545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:47.424232 17545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:47.424248 17545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:47.430401 17545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:45969,127.8.49.61:38683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.61.keytab
--principal=oryx/127.8.49.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.61:38683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.49.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
W20260806 16:06:47.430711 17545 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.49.62:45969,127.8.49.61:38683'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 16:06:47.431773 17545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:47.432946 17545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:06:47.443239 17545 server_base.cc:1034] running on GCE node
W20260806 16:06:47.443437 17556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:47.443303 17559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:47.443303 17554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:47.444485 17545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:47.445777 17545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:47.447006 17545 hybrid_clock.cc:648] HybridClock initialized: now 1786032407446972 us; error 60 us; skew 500 ppm
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:47.450730 17545 init.cc:377] Logged in from keytab as oryx/127.8.49.61@KRBTEST.COM (short username oryx)
I20260806 16:06:47.452103 17545 webserver.cc:460] Webserver started at http://127.8.49.61:43289/ using document root <none> and password file <none>
I20260806 16:06:47.452809 17545 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:47.452924 17545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:47.453187 17545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:47.455777 17545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data/instance:
uuid: "6274597ce8be486fba4a0613d099ea9f"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "8c4b66a25fd25156ce65d07af67014d8"
server_key_iv: "d07f987786830379efb6572e92edcd13"
server_key_version: "encryptionkey@0"
I20260806 16:06:47.456470 17545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/wal/instance:
uuid: "6274597ce8be486fba4a0613d099ea9f"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "8c4b66a25fd25156ce65d07af67014d8"
server_key_iv: "d07f987786830379efb6572e92edcd13"
server_key_version: "encryptionkey@0"
I20260806 16:06:47.461362 17545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:06:47.465086 17571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:47.466934 17545 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:47.467155 17545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/wal
uuid: "6274597ce8be486fba4a0613d099ea9f"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "8c4b66a25fd25156ce65d07af67014d8"
server_key_iv: "d07f987786830379efb6572e92edcd13"
server_key_version: "encryptionkey@0"
I20260806 16:06:47.467306 17545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:47.483368 17545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:47.487085 17545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:47.487330 17545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:47.495867 17545 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.61:38683
I20260806 16:06:47.496338 17649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.61:38683 every 8 connection(s)
I20260806 16:06:47.497108 17545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
I20260806 16:06:47.501384 17650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:47.502143 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 17545
I20260806 16:06:47.509467 17650 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.49.61@KRBTEST.COM
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.61@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
I20260806 16:06:47.524453 17656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.504524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45754 (local address 127.8.49.61:38683)
0806 16:06:47.505252 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.505276 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.505327 (+ 51us) server_negotiation.cc:407] Connection header received
0806 16:06:47.506373 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.506458 (+ 85us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.507027 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.507460 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.508555 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.509862 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.511075 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.511962 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.512161 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.516344 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.516351 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.516365 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.516408 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.519100 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.519819 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.519832 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.519844 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.520066 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.520470 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.520478 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.520481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.521277 (+ 796us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.521770 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.522285 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.523146 (+ 861us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":210,"threads_started":1}
I20260806 16:06:47.524922 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.512597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52176 (local address 127.8.49.62:45969)
0806 16:06:47.512851 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.512864 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.513363 (+ 499us) server_negotiation.cc:407] Connection header received
0806 16:06:47.513553 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.513557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.513640 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.513785 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.514739 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.515360 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.517027 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.517263 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.517493 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.520580 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.520584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.520586 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.520618 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.522589 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.523349 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.523353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.523355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.523453 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.523775 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.523779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.523781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.524041 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.524163 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.524575 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.524723 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260806 16:06:47.524979 17657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.512513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:52176)
0806 16:06:47.513180 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:47.513241 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:47.513412 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:47.513874 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:47.513880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:47.514301 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:47.514467 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.514474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.515595 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.515604 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.516799 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.516820 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.517492 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.517497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.517672 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.518210 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:47.518223 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:47.520359 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:47.522732 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.522745 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.522756 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.523224 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.523557 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.523560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.523562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.523675 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.524164 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:47.524175 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:47.524345 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 16:06:47.524514 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 16:06:47.524760 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":221,"threads_started":1}
I20260806 16:06:47.526571 17650 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.61@KRBTEST.COM for oryx/127.8.49.61@KRBTEST.COM
I20260806 16:06:47.538708 17656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.527434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.8.49.61:38683)
0806 16:06:47.527560 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.527567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.527582 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:06:47.527743 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.527746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.527814 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.527902 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.528627 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.529653 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.530748 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.531152 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.531275 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.534141 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.534145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.534147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.534182 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.536031 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.536744 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.536750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.536752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.536886 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.537337 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.537342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.537346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.537675 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.537808 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.538237 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.538470 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:06:47.538731 17657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.527321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:38683 (local address 127.0.0.1:45768)
0806 16:06:47.527521 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:47.527538 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:47.527645 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:47.527979 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:47.527982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:47.528267 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:47.528416 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.528422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.529823 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.529826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.530514 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.530532 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.531415 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.531418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.531523 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.532221 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:47.532234 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:47.533928 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:47.536176 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.536179 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.536181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.536579 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.537042 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.537048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.537052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.537199 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.537814 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:47.537817 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:47.537982 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 16:06:47.538083 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 16:06:47.538254 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 16:06:47.542812 17650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: Bootstrap starting.
I20260806 16:06:47.546079 17650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:47.547173 17650 log.cc:824] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:47.550470 17650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: No bootstrap required, opened a new log
I20260806 16:06:47.554888 17650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } }
I20260806 16:06:47.555253 17650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:47.555295 17650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6274597ce8be486fba4a0613d099ea9f, State: Initialized, Role: FOLLOWER
I20260806 16:06:47.555842 17650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } }
I20260806 16:06:47.556643 17660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } } }
I20260806 16:06:47.556931 17660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:06:47.557572 17650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: configured and running, proceeding with master startup.
W20260806 16:06:47.565109 17675 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:06:47.565251 17675 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:06:47.569680 17545 master_runner.cc:428] Detected that this master 6274597ce8be486fba4a0613d099ea9f is joining an existing cluster
I20260806 16:06:47.569767 17545 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.49.61:38683
I20260806 16:06:47.587800 17656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.573168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45774 (local address 127.8.49.61:38683)
0806 16:06:47.573326 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.573332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.573487 (+ 155us) server_negotiation.cc:407] Connection header received
0806 16:06:47.573705 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.573709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.573780 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.573895 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.574453 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.575357 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.576674 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.577053 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.577198 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.579337 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.579342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.579344 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.579369 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.584800 (+ 5431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.586186 (+ 1386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.586192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.586194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.586306 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.586667 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.586670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.586672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.586999 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.587081 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.587389 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.587569 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":58}
I20260806 16:06:47.588018 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.572660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52180 (local address 127.8.49.62:45969)
0806 16:06:47.572806 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.572811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.573137 (+ 326us) server_negotiation.cc:407] Connection header received
0806 16:06:47.573366 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.573371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.573439 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.573500 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.574329 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.575179 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.576673 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.576908 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.576980 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.579348 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.579352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.579355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.579383 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.585208 (+ 5825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.586140 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.586146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.586148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.586261 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.586687 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.586694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.586698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.587039 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.587133 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.587581 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.587746 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:06:47.588778 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.572593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:52180)
0806 16:06:47.573032 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:47.573051 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:47.573233 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:47.573696 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:47.573699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:47.573983 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:47.574172 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.574181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.575308 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.575311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.576521 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.576533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.577186 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.577190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.577425 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.578415 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:47.578433 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:47.579176 (+ 743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:47.585343 (+ 6167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.585348 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.585354 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.585986 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.586400 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.586404 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.586408 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.586559 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.587227 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:47.587232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:47.587369 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 16:06:47.588246 (+ 877us) client_negotiation.cc:240] Negotiation successful
0806 16:06:47.588469 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":154,"spinlock_wait_cycles":3200,"thread_start_us":135,"threads_started":1}
I20260806 16:06:47.589314 17686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.572749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:38683 (local address 127.0.0.1:45774)
0806 16:06:47.573365 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:47.573379 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:47.573494 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:47.573939 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:47.573941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:47.574167 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:47.574307 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.574315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.575498 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.575501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.576523 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.576533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.577187 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.577190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.577432 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.578551 (+ 1119us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:47.578570 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:47.579177 (+ 607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:47.585017 (+ 5840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.585024 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.585027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.586008 (+ 981us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.586422 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.586425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.586427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.586570 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.587141 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:47.587144 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:47.587230 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 16:06:47.588634 (+ 1404us) client_negotiation.cc:240] Negotiation successful
0806 16:06:47.589102 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":151,"spinlock_wait_cycles":7296,"thread_start_us":150,"threads_started":1}
WARNING: no policy specified for oryx/127.8.49.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.60@KRBTEST.COM" created.
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032406, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.62@KRBTEST.COM for oryx/127.8.49.61@KRBTEST.COM
I20260806 16:06:47.608731 17656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.595926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45778 (local address 127.8.49.61:38683)
0806 16:06:47.596091 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.596115 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.596600 (+ 485us) server_negotiation.cc:407] Connection header received
0806 16:06:47.596864 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.596867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.596932 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.597033 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.597757 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.598465 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.599924 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.600353 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.600525 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.603453 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.603457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.603459 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.603490 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.605220 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.606064 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.606071 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.606073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.606266 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.606761 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.606765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.606767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.607249 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.607396 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.608096 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.608432 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:06:47.609683 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.595886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:38683 (local address 127.0.0.1:45778)
0806 16:06:47.596396 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:47.596485 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:47.596700 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:47.597077 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:47.597084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:47.597496 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:47.597625 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.597631 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.598591 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.598594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.599717 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.599731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.600560 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.600568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.600727 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.601559 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:47.601583 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:47.603246 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:47.605407 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.605425 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.605439 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.605861 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.606461 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.606468 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.606474 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.606619 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.607423 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:47.607436 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:47.607689 (+ 253us) client_negotiation.cc:769] Sending connection context
0806 16:06:47.609045 (+ 1356us) client_negotiation.cc:240] Negotiation successful
0806 16:06:47.609372 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":167,"threads_started":1}
I20260806 16:06:47.612201 16871 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.49.61:38683: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6274597ce8be486fba4a0613d099ea9f"
member_type: NON_VOTER
last_known_addr {
host: "127.8.49.61"
port: 38683
}
attrs {
promote: true
}
}
dest_uuid: "62f1112cc8e447e19ce70e0eab5a186b"
cas_config_opid_index: -1
I20260806 16:06:47.613230 16871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } }
I20260806 16:06:47.615923 17696 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6274597ce8be486fba4a0613d099ea9f (127.8.49.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } } }
I20260806 16:06:47.617228 17610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:47.617998 17696 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.49.61:38683
I20260806 16:06:47.618301 17696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } } }
I20260806 16:06:47.618487 17696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:47.619606 17545 master.cc:537] Master@127.8.49.61:38683 shutting down...
I20260806 16:06:47.623133 17610 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Refusing update from remote peer 62f1112cc8e447e19ce70e0eab5a186b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 16:06:47.624697 17695 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 16:06:47.629660 16846 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.60.keytab.
Entry for principal oryx/127.8.49.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.60.keytab.
W20260806 16:06:47.631629 16846 consensus_peers.cc:487] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b -> Peer 6274597ce8be486fba4a0613d099ea9f (127.8.49.61:38683): Couldn't send request to peer 6274597ce8be486fba4a0613d099ea9f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 16:06:47.632767 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.60:39957
--webserver_interface=127.8.49.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.60.keytab
--principal=oryx/127.8.49.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:45969,127.8.49.61:38683,127.8.49.60:39957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:06:47.633100 17545 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:47.633673 17545 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:47.633868 17545 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: stopping tablet replica
I20260806 16:06:47.643100 17545 master.cc:552] Master@127.8.49.61:38683 shutdown complete.
I20260806 16:06:47.643203 17545 master_runner.cc:305] Clearing existing system tablet
I20260806 16:06:47.643548 17545 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:47.643618 17545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:47.645411 17545 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 16:06:47.646992 17703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:47.647509 17545 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:47.647778 17545 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:47.647873 17545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/wal
uuid: "6274597ce8be486fba4a0613d099ea9f"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "8c4b66a25fd25156ce65d07af67014d8"
server_key_iv: "d07f987786830379efb6572e92edcd13"
server_key_version: "encryptionkey@0"
I20260806 16:06:47.651312 17545 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:47.656061 17545 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 16:06:47.656323 17545 log.cc:1196] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 16:06:47.656728 17545 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: Deleting consensus metadata
I20260806 16:06:47.657004 17545 master_runner.cc:315] Copying system tablet from 127.8.49.62:45969
I20260806 16:06:47.659138 17144 heartbeater.cc:503] Master 127.8.49.62:45969 was elected leader, sending a full tablet report...
I20260806 16:06:47.660274 17545 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: tablet copy: Beginning tablet copy session from remote peer at address 127.8.49.62:45969
I20260806 16:06:47.676331 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.663601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52184 (local address 127.8.49.62:45969)
0806 16:06:47.663787 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.663793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.664503 (+ 710us) server_negotiation.cc:407] Connection header received
0806 16:06:47.664674 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.664679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.664764 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.664917 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.665624 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.666518 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.667922 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.668300 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.668437 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.670159 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.670164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.670167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.670206 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.672895 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.673839 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.673844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.673846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.673982 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.674390 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.674394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.674396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.674795 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.675361 (+ 566us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.675859 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.676076 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 16:06:47.677628 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.663836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:52184)
0806 16:06:47.664324 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:47.664344 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:47.664528 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:47.664987 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:47.664990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:47.665275 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:47.665445 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.665454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.666692 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.666696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.667750 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.667765 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.668470 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.668476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.668587 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.669351 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:47.669367 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:47.669915 (+ 548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:47.673087 (+ 3172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.673091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.673094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.673528 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.674121 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.674124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.674126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.674254 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.675444 (+ 1190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:47.675449 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:47.675614 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 16:06:47.677121 (+ 1507us) client_negotiation.cc:240] Negotiation successful
0806 16:06:47.677383 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":181,"threads_started":1}
I20260806 16:06:47.679255 16898 tablet_copy_service.cc:140] P 62f1112cc8e447e19ce70e0eab5a186b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6274597ce8be486fba4a0613d099ea9f ({username='oryx', principal='oryx/127.8.49.61@KRBTEST.COM'} at 127.0.0.1:52184)
I20260806 16:06:47.679589 16898 tablet_copy_service.cc:161] P 62f1112cc8e447e19ce70e0eab5a186b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6274597ce8be486fba4a0613d099ea9f at {username='oryx', principal='oryx/127.8.49.61@KRBTEST.COM'} at 127.0.0.1:52184: session id = 6274597ce8be486fba4a0613d099ea9f-00000000000000000000000000000000
I20260806 16:06:47.682144 16898 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 16:06:47.684964 17545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 16:06:47.688354 17545 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: tablet copy: Starting download of 0 data blocks...
I20260806 16:06:47.688653 17545 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: tablet copy: Starting download of 1 WAL segments...
I20260806 16:06:47.691591 17545 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 16:06:47.694790 16898 tablet_copy_service.cc:342] P 62f1112cc8e447e19ce70e0eab5a186b: Request end of tablet copy session 6274597ce8be486fba4a0613d099ea9f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.49.61@KRBTEST.COM'} at 127.0.0.1:52184
I20260806 16:06:47.695106 16898 tablet_copy_service.cc:433] P 62f1112cc8e447e19ce70e0eab5a186b: ending tablet copy session 6274597ce8be486fba4a0613d099ea9f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6274597ce8be486fba4a0613d099ea9f
I20260806 16:06:47.708009 17545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:47.710685 17713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:47.711365 17715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:47.711400 17719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:47.712913 17545 server_base.cc:1034] running on GCE node
I20260806 16:06:47.713436 17545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:47.713940 17545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:47.715107 17545 hybrid_clock.cc:648] HybridClock initialized: now 1786032407715075 us; error 61 us; skew 500 ppm
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:47.717305 17545 init.cc:377] Logged in from keytab as oryx/127.8.49.61@KRBTEST.COM (short username oryx)
I20260806 16:06:47.718017 17545 webserver.cc:460] Webserver started at http://127.8.49.61:35001/ using document root <none> and password file <none>
I20260806 16:06:47.718325 17545 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:47.718423 17545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:47.720525 17545 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 16:06:47.721961 17732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:47.722406 17545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 16:06:47.722558 17545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/wal
uuid: "6274597ce8be486fba4a0613d099ea9f"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "8c4b66a25fd25156ce65d07af67014d8"
server_key_iv: "d07f987786830379efb6572e92edcd13"
server_key_version: "encryptionkey@0"
I20260806 16:06:47.722894 17545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:47.739418 17545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:47.742380 17545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:47.750197 17545 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.61:38683
I20260806 16:06:47.750250 17804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.61:38683 every 8 connection(s)
I20260806 16:06:47.751628 17545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
I20260806 16:06:47.754303 17805 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:06:47.755697 17805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: Bootstrap starting.
I20260806 16:06:47.768764 17805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:06:47.769194 17805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: Bootstrap complete.
I20260806 16:06:47.770140 17805 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } }
I20260806 16:06:47.770326 17805 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6274597ce8be486fba4a0613d099ea9f, State: Initialized, Role: LEARNER
I20260806 16:06:47.770502 17805 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } }
I20260806 16:06:47.771109 17805 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:47.771282 17809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } } }
I20260806 16:06:47.771736 17809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: This master's current role is: LEARNER
I20260806 16:06:47.775234 17830 catalog_manager.cc:1259] Loaded cluster ID: 0b8b98af43d743aa824a44346403a0cc
I20260806 16:06:47.775323 17830 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: loading cluster ID for follower catalog manager: success
I20260806 16:06:47.778223 17830 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: acquiring CA information for follower catalog manager: success
I20260806 16:06:47.780122 17830 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 16:06:47.796428 17701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:47.796801 17701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:47.796850 17701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:47.801666 17701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:47.801762 17701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:47.801783 17701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:47.801801 17701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:47.801823 17701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:47.808029 17701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:45969,127.8.49.61:38683,127.8.49.60:39957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.60.keytab
--principal=oryx/127.8.49.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.60:39957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.49.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:47.809505 17701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:47.810814 17701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:47.821615 17840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:47.821632 17838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:47.821497 17843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:47.822002 17701 server_base.cc:1034] running on GCE node
I20260806 16:06:47.822961 17701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:47.824375 17701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:47.825603 17701 hybrid_clock.cc:648] HybridClock initialized: now 1786032407825575 us; error 66 us; skew 500 ppm
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:47.829417 17701 init.cc:377] Logged in from keytab as oryx/127.8.49.60@KRBTEST.COM (short username oryx)
I20260806 16:06:47.830639 17701 webserver.cc:460] Webserver started at http://127.8.49.60:44297/ using document root <none> and password file <none>
I20260806 16:06:47.831401 17701 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:47.831487 17701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:47.831736 17701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:47.834177 17701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data/instance:
uuid: "939100169d094eeba3e04bb2ea5304ff"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "435cbb1dfd93b5d13669dd26e527dcd9"
server_key_iv: "1ab1670d6240d1f5e8d447d635beff68"
server_key_version: "encryptionkey@0"
I20260806 16:06:47.834888 17701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/wal/instance:
uuid: "939100169d094eeba3e04bb2ea5304ff"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "435cbb1dfd93b5d13669dd26e527dcd9"
server_key_iv: "1ab1670d6240d1f5e8d447d635beff68"
server_key_version: "encryptionkey@0"
I20260806 16:06:47.839699 17701 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 16:06:47.843145 17851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:47.845459 17701 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:06:47.845676 17701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/wal
uuid: "939100169d094eeba3e04bb2ea5304ff"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "435cbb1dfd93b5d13669dd26e527dcd9"
server_key_iv: "1ab1670d6240d1f5e8d447d635beff68"
server_key_version: "encryptionkey@0"
I20260806 16:06:47.845877 17701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:47.868700 17701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:47.873720 17701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:47.874045 17701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:47.884443 17701 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.60:39957
I20260806 16:06:47.884524 17932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.60:39957 every 8 connection(s)
I20260806 16:06:47.885679 17701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
I20260806 16:06:47.890450 17933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:47.895489 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 17701
I20260806 16:06:47.896950 17933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.60@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.49.60@KRBTEST.COM
I20260806 16:06:47.915539 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.899902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52200 (local address 127.8.49.62:45969)
0806 16:06:47.900059 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.900064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.900941 (+ 877us) server_negotiation.cc:407] Connection header received
0806 16:06:47.901199 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.901203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.901268 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.901334 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.902789 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.903460 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.905067 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.905437 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.905598 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.909648 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.909655 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.909657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.909701 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.912041 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.913056 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.913059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.913061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.913155 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.913808 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.913814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.913818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.914143 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.914272 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.915133 (+ 861us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.915319 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 16:06:47.916877 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.899715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:52200)
0806 16:06:47.900751 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:47.900809 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:47.901054 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:47.901612 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:47.901619 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:47.902207 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:47.902612 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.902627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.903624 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.903628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.904863 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.904876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.905674 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.905680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.905884 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.907068 (+ 1184us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:47.907091 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:47.909288 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:47.912271 (+ 2983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.912289 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.912303 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.912879 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.913272 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.913276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.913279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.913627 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.914285 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:47.914293 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:47.914764 (+ 471us) client_negotiation.cc:769] Sending connection context
0806 16:06:47.915071 (+ 307us) client_negotiation.cc:240] Negotiation successful
0806 16:06:47.915567 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":60,"spinlock_wait_cycles":1920,"thread_start_us":124,"threads_started":1}
I20260806 16:06:47.916874 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.898041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44946 (local address 127.8.49.60:39957)
0806 16:06:47.898806 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.898872 (+ 66us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.898944 (+ 72us) server_negotiation.cc:407] Connection header received
0806 16:06:47.900163 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.900198 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.900769 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.901198 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.902430 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.904010 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.905242 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.905713 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.905897 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.910155 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.910164 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.910185 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.910241 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.912864 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.913556 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.913563 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.913565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.913666 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.914103 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.914107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.914109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.914737 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.915002 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.915409 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.915661 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":530,"thread_start_us":246,"threads_started":1}
I20260806 16:06:47.919070 17933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.60@KRBTEST.COM for oryx/127.8.49.61@KRBTEST.COM
I20260806 16:06:47.930517 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.919984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45794 (local address 127.8.49.61:38683)
0806 16:06:47.920378 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.920382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.920396 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:06:47.920447 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.920450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.920513 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.920617 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.921266 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.922011 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.923068 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.923448 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.923650 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.926435 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.926439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.926442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.926488 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.928477 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.929028 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.929033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.929035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.929132 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.929453 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.929456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.929458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.929718 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.929844 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.930131 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.930295 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":199,"threads_started":1}
I20260806 16:06:47.930496 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.919814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:38683 (local address 127.0.0.1:45794)
0806 16:06:47.920085 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:47.920099 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:47.920177 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:47.920734 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:47.920738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:47.920925 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:47.921085 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.921092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.922159 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.922162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.922814 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.922904 (+ 90us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.923715 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.923720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.923868 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.924489 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:47.924502 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:47.926238 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:47.928655 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.928659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.928664 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.928909 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.929245 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.929249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.929251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.929341 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.929829 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:47.929832 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:47.929951 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 16:06:47.930068 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 16:06:47.930278 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260806 16:06:47.932103 17933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:06:47.932873 17340 heartbeater.cc:503] Master 127.8.49.62:45969 was elected leader, sending a full tablet report...
Aug 06 16:06:47 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032407, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.60@KRBTEST.COM for oryx/127.8.49.60@KRBTEST.COM
I20260806 16:06:47.943624 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.932976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:39957 (local address 127.0.0.1:44962)
0806 16:06:47.933263 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:47.933278 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:47.933378 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:47.933697 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:47.933701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:47.933945 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:47.934068 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.934075 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.935611 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.935614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.936222 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:47.936232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.936872 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.936875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:47.937004 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.937658 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:47.937672 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:47.939447 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:47.941616 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.941620 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.941621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.941938 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.942327 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:47.942330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:47.942331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:47.942442 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:47.943066 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:47.943069 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:47.943193 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 16:06:47.943302 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 16:06:47.943465 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260806 16:06:47.943951 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.933028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44962 (local address 127.8.49.60:39957)
0806 16:06:47.933221 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.933226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:47.933359 (+ 133us) server_negotiation.cc:407] Connection header received
0806 16:06:47.933483 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:47.933487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:47.933549 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:47.933638 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:47.934218 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.935468 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.936395 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:47.936661 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:47.936834 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:47.939648 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:47.939654 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:47.939657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:47.939695 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:47.941433 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.942079 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.942082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.942084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.942219 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:47.942567 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:47.942570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:47.942572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:47.942952 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:47.943051 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:47.943426 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 16:06:47.943577 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 16:06:47.948225 17933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: Bootstrap starting.
I20260806 16:06:47.951664 17933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:47.952638 17933 log.cc:824] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:47.955855 17933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: No bootstrap required, opened a new log
I20260806 16:06:47.959823 17933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } }
I20260806 16:06:47.960125 17933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:47.960163 17933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939100169d094eeba3e04bb2ea5304ff, State: Initialized, Role: FOLLOWER
I20260806 16:06:47.960764 17933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } }
I20260806 16:06:47.961544 17939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } } }
I20260806 16:06:47.961735 17939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:06:47.963299 17933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: configured and running, proceeding with master startup.
W20260806 16:06:47.971828 17954 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:06:47.971968 17954 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:06:47.972807 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 16964
I20260806 16:06:47.996212 17701 master_runner.cc:428] Detected that this master 939100169d094eeba3e04bb2ea5304ff is joining an existing cluster
I20260806 16:06:47.996354 17701 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.49.60:39957
I20260806 16:06:48.006320 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:40541
--local_ip_for_outbound_sockets=127.8.49.1
--tserver_master_addrs=127.8.49.62:45969
--webserver_port=39373
--webserver_interface=127.8.49.1
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab
--principal=oryx/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.49.62:45969,127.8.49.61:38683
--tserver_master_addrs=127.8.49.62:45969,127.8.49.61:38683,127.8.49.60:39957 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:06:48.016099 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.000835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45808 (local address 127.8.49.61:38683)
0806 16:06:48.000998 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.001005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.002047 (+ 1042us) server_negotiation.cc:407] Connection header received
0806 16:06:48.002255 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.002260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.002327 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.002386 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.003235 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.004123 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.005373 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.005735 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.005819 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.011738 (+ 5919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.011744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.011746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.011785 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.013796 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.014536 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.014541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.014543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.014652 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.015037 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.015041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.015042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.015304 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.015426 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.015747 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.015905 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 16:06:48.016120 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.000691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:38683 (local address 127.0.0.1:45808)
0806 16:06:48.001954 (+ 1263us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.001969 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.002111 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.002583 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.002585 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.002912 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.003065 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.003072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.004262 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.004265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.005225 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.005235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.006052 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.006055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.006177 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.010815 (+ 4638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.010886 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.011503 (+ 617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.013968 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.013973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.013976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.014347 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.014764 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.014767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.014769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.014931 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.015427 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.015430 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.015549 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.015659 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.015872 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1172,"mutex_wait_us":9,"thread_start_us":109,"threads_started":1}
I20260806 16:06:48.048108 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.999748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44968 (local address 127.8.49.60:39957)
0806 16:06:47.999925 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.999932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.001790 (+ 1858us) server_negotiation.cc:407] Connection header received
0806 16:06:48.001831 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.001833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.001870 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.001946 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.003309 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.004121 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.008016 (+ 3895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.008382 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.008468 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.042710 (+ 34242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.042715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.042718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.042750 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.045743 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.046417 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.046423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.046428 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.046520 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.046851 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.046857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.046861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.047175 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.047263 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.047654 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.047842 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 16:06:48.048529 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:47.999739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52206 (local address 127.8.49.62:45969)
0806 16:06:47.999970 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:47.999978 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.001267 (+ 1289us) server_negotiation.cc:407] Connection header received
0806 16:06:48.001455 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.001460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.001534 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.001634 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.004354 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.005212 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.007467 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.007903 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.008007 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.042765 (+ 34758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.042780 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.042783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.042861 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.046346 (+ 3485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.046934 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.046938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.046939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.047037 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.047366 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.047369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.047371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.047636 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.047722 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.048058 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.048241 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 16:06:48.048734 17965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.000127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:39957 (local address 127.0.0.1:44968)
0806 16:06:48.001579 (+ 1452us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.001594 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.001697 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.002881 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.002884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.003065 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.003194 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.003204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.004262 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.004265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.007861 (+ 3596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.007877 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.008656 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.008659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.008753 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.039992 (+ 31239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.040028 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.042497 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.045984 (+ 3487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.045990 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.045995 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.046298 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.046620 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.046623 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.046625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.046723 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.047314 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.047319 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.047461 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.048355 (+ 894us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.048524 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1384,"mutex_wait_us":2283,"thread_start_us":121,"threads_started":1}
I20260806 16:06:48.050200 17966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.000033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:52206)
0806 16:06:48.001141 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.001162 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.001323 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.002082 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.002084 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.002436 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.002602 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.002610 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.005381 (+ 2771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.005384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.006421 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.006437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.008271 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.008275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.008382 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.040717 (+ 32335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.040739 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.042372 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.046492 (+ 4120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.046496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.046498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.046768 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.047169 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.047172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.047174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.047257 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.047781 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.047786 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.047872 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.048769 (+ 897us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.049692 (+ 923us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"spinlock_wait_cycles":3840,"thread_start_us":175,"threads_started":1}
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032406, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.62@KRBTEST.COM for oryx/127.8.49.60@KRBTEST.COM
I20260806 16:06:48.073520 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.056470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:39957 (local address 127.0.0.1:44984)
0806 16:06:48.056714 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.056733 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.056856 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.057237 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.057240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.057526 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.057653 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.057661 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.058694 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.058698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.059412 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.059423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.059951 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.059955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.060068 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.060748 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.060762 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.062548 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.065845 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.066047 (+ 202us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.066050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.068044 (+ 1994us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.068726 (+ 682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.068730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.068732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.069215 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.071798 (+ 2583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.072106 (+ 308us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.072656 (+ 550us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.072873 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.073275 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260806 16:06:48.073572 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.056549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44984 (local address 127.8.49.60:39957)
0806 16:06:48.056717 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.056723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.056786 (+ 63us) server_negotiation.cc:407] Connection header received
0806 16:06:48.057042 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.057045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.057107 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.057214 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.057792 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.058528 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.059548 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.059802 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.059928 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.062885 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.062891 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.062894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.062937 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.065428 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.068275 (+ 2847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.068282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.068284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.068566 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.069483 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.069488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.069491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.071401 (+ 1910us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.072084 (+ 683us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.072981 (+ 897us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.073289 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:06:48.075870 16871 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.49.60:39957: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "939100169d094eeba3e04bb2ea5304ff"
member_type: NON_VOTER
last_known_addr {
host: "127.8.49.60"
port: 39957
}
attrs {
promote: true
}
}
dest_uuid: "62f1112cc8e447e19ce70e0eab5a186b"
cas_config_opid_index: 5
I20260806 16:06:48.077425 16871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } }
I20260806 16:06:48.079128 17695 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 939100169d094eeba3e04bb2ea5304ff (127.8.49.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.079946 17897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:48.081792 17695 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.49.60:39957
I20260806 16:06:48.082355 17897 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Refusing update from remote peer 62f1112cc8e447e19ce70e0eab5a186b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 16:06:48.083422 17701 master.cc:537] Master@127.8.49.60:39957 shutting down...
I20260806 16:06:48.083953 17968 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 16:06:48.087342 16844 consensus_peers.cc:487] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b -> Peer 939100169d094eeba3e04bb2ea5304ff (127.8.49.60:39957): Couldn't send request to peer 939100169d094eeba3e04bb2ea5304ff. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 16:06:48.095964 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.087357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45824 (local address 127.8.49.61:38683)
0806 16:06:48.090436 (+ 3079us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.090443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.090472 (+ 29us) server_negotiation.cc:407] Connection header received
0806 16:06:48.090592 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.090595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.090688 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.090899 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:48.091322 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.092159 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.093797 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.094730 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.094873 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.095146 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.095635 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.095766 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2779}
I20260806 16:06:48.096745 17696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.096946 17696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:48.097100 17939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:48.097328 17939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } }
I20260806 16:06:48.097649 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.086563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:38683 (local address 127.0.0.1:45824)
0806 16:06:48.087955 (+ 1392us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.087979 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.088106 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.090959 (+ 2853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.090963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.090991 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:48.091142 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.091151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.092349 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.092352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.093600 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.093613 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.095283 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.095286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.095403 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.095478 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.096315 (+ 837us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.097508 (+ 1193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1233}
I20260806 16:06:48.098416 17771 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 LEARNER]: Refusing update from remote peer 62f1112cc8e447e19ce70e0eab5a186b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 16:06:48.098694 17939 leader_election.cc:288] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969), 6274597ce8be486fba4a0613d099ea9f (127.8.49.61:38683)
I20260806 16:06:48.099471 17771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "939100169d094eeba3e04bb2ea5304ff" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6274597ce8be486fba4a0613d099ea9f" is_pre_election: true
I20260806 16:06:48.099690 17771 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 LEARNER]: Handling vote request from an unknown peer 939100169d094eeba3e04bb2ea5304ff
I20260806 16:06:48.099833 17771 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 939100169d094eeba3e04bb2ea5304ff for term 2 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260806 16:06:48.101099 17701 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:48.101107 16885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "939100169d094eeba3e04bb2ea5304ff" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62f1112cc8e447e19ce70e0eab5a186b" is_pre_election: true
I20260806 16:06:48.101586 17701 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:48.101626 17696 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 16:06:48.101719 17701 tablet_replica.cc:330] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: stopping tablet replica
I20260806 16:06:48.102331 17856 leader_election.cc:302] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 939100169d094eeba3e04bb2ea5304ff; no voters: 6274597ce8be486fba4a0613d099ea9f, 62f1112cc8e447e19ce70e0eab5a186b
W20260806 16:06:48.102624 17856 raft_consensus.cc:2670] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260806 16:06:48.105450 17771 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 939100169d094eeba3e04bb2ea5304ff (127.8.49.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.111222 17809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62f1112cc8e447e19ce70e0eab5a186b. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.111445 17809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: This master's current role is: LEARNER
I20260806 16:06:48.111332 17973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: NON_VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: true } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.111677 17973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: This master's current role is: LEARNER
I20260806 16:06:48.112147 17969 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b: attempting to promote NON_VOTER 6274597ce8be486fba4a0613d099ea9f to VOTER
I20260806 16:06:48.112807 17969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } }
I20260806 16:06:48.114411 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.114043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:39957 (local address 127.0.0.1:44998)
0806 16:06:48.114191 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.114294 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:39957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 16:06:48.115020 17771 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 LEARNER]: Refusing update from remote peer 62f1112cc8e447e19ce70e0eab5a186b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 16:06:48.115784 17696 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 16:06:48.115911 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.115349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:39957 (local address 127.0.0.1:45006)
0806 16:06:48.115766 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.115829 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:39957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329}
W20260806 16:06:48.116017 16844 consensus_peers.cc:487] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b -> Peer 939100169d094eeba3e04bb2ea5304ff (127.8.49.60:39957): Couldn't send request to peer 939100169d094eeba3e04bb2ea5304ff. Status: Network error: Client connection negotiation failed: client connection to 127.8.49.60:39957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 16:06:48.117872 17696 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6274597ce8be486fba4a0613d099ea9f (127.8.49.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.119802 17696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.119930 17696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:48.119963 17968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.120064 17968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:48.121241 17771 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6274597ce8be486fba4a0613d099ea9f (127.8.49.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.122105 17701 master.cc:552] Master@127.8.49.60:39957 shutdown complete.
I20260806 16:06:48.122156 17701 master_runner.cc:305] Clearing existing system tablet
I20260806 16:06:48.122452 17701 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:48.122504 17701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:48.124320 17973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.124480 17973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:06:48.124985 17701 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:48.126210 17975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:48.126478 17701 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:48.126626 17701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 16:06:48.126663 17701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/wal
uuid: "939100169d094eeba3e04bb2ea5304ff"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "435cbb1dfd93b5d13669dd26e527dcd9"
server_key_iv: "1ab1670d6240d1f5e8d447d635beff68"
server_key_version: "encryptionkey@0"
I20260806 16:06:48.129048 17701 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:48.132165 17701 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 16:06:48.132362 17701 log.cc:1196] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 16:06:48.132741 17701 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: Deleting consensus metadata
I20260806 16:06:48.133309 17701 master_runner.cc:315] Copying system tablet from 127.8.49.62:45969
I20260806 16:06:48.135010 17701 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: tablet copy: Beginning tablet copy session from remote peer at address 127.8.49.62:45969
I20260806 16:06:48.145609 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.135844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:52214)
0806 16:06:48.136350 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.136364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.136541 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.136856 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.136859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.137120 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.137233 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.137239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.138081 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.138085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.138728 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.138737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.139319 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.139323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.139429 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.140120 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.140133 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.140555 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.143344 (+ 2789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.143349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.143352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.143650 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.144103 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.144108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.144110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.144214 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.144849 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.144853 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.145000 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.145147 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.145384 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":119,"threads_started":1}
I20260806 16:06:48.145609 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.135921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52214 (local address 127.8.49.62:45969)
0806 16:06:48.136098 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.136103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.136466 (+ 363us) server_negotiation.cc:407] Connection header received
0806 16:06:48.136673 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.136676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.136733 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.136835 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.137364 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.137957 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.138888 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.139174 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.139304 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.140738 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.140743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.140745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.140772 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.143039 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.143868 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.143873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.143875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.143986 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.144339 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.144344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.144346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.144691 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.144911 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.145212 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.145378 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 16:06:48.146780 16898 tablet_copy_service.cc:140] P 62f1112cc8e447e19ce70e0eab5a186b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 939100169d094eeba3e04bb2ea5304ff ({username='oryx', principal='oryx/127.8.49.60@KRBTEST.COM'} at 127.0.0.1:52214)
I20260806 16:06:48.146978 16898 tablet_copy_service.cc:161] P 62f1112cc8e447e19ce70e0eab5a186b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 939100169d094eeba3e04bb2ea5304ff at {username='oryx', principal='oryx/127.8.49.60@KRBTEST.COM'} at 127.0.0.1:52214: session id = 939100169d094eeba3e04bb2ea5304ff-00000000000000000000000000000000
I20260806 16:06:48.148167 16898 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 16:06:48.150067 17701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 16:06:48.152896 17701 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: tablet copy: Starting download of 0 data blocks...
I20260806 16:06:48.153398 17701 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: tablet copy: Starting download of 1 WAL segments...
I20260806 16:06:48.156314 17701 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 16:06:48.159962 16898 tablet_copy_service.cc:342] P 62f1112cc8e447e19ce70e0eab5a186b: Request end of tablet copy session 939100169d094eeba3e04bb2ea5304ff-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.49.60@KRBTEST.COM'} at 127.0.0.1:52214
I20260806 16:06:48.160126 16898 tablet_copy_service.cc:433] P 62f1112cc8e447e19ce70e0eab5a186b: ending tablet copy session 939100169d094eeba3e04bb2ea5304ff-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 939100169d094eeba3e04bb2ea5304ff
I20260806 16:06:48.172737 17701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:48.175782 17989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:48.175716 17985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:48.176077 17984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:48.176028 17701 server_base.cc:1034] running on GCE node
I20260806 16:06:48.176911 17701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:48.177620 17701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:48.178948 17701 hybrid_clock.cc:648] HybridClock initialized: now 1786032408178936 us; error 46 us; skew 500 ppm
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:48.180874 17701 init.cc:377] Logged in from keytab as oryx/127.8.49.60@KRBTEST.COM (short username oryx)
I20260806 16:06:48.181819 17701 webserver.cc:460] Webserver started at http://127.8.49.60:39345/ using document root <none> and password file <none>
I20260806 16:06:48.182199 17701 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:48.182271 17701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:48.183844 17701 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 16:06:48.185060 18001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:48.185379 17701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 16:06:48.185487 17701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/wal
uuid: "939100169d094eeba3e04bb2ea5304ff"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "435cbb1dfd93b5d13669dd26e527dcd9"
server_key_iv: "1ab1670d6240d1f5e8d447d635beff68"
server_key_version: "encryptionkey@0"
I20260806 16:06:48.185621 17701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:48.194387 17701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:48.197443 17701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:48.206708 17701 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.60:39957
I20260806 16:06:48.208060 17701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
I20260806 16:06:48.216784 18080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.60:39957 every 8 connection(s)
I20260806 16:06:48.218441 18082 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:06:48.220544 18082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: Bootstrap starting.
I20260806 16:06:48.237807 17538 heartbeater.cc:503] Master 127.8.49.62:45969 was elected leader, sending a full tablet report...
I20260806 16:06:48.243682 18082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:06:48.244455 18082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: Bootstrap complete.
I20260806 16:06:48.245440 18082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } }
I20260806 16:06:48.246088 18082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 939100169d094eeba3e04bb2ea5304ff, State: Initialized, Role: LEARNER
I20260806 16:06:48.246557 18082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } }
I20260806 16:06:48.248168 18082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: configured and running, proceeding with master startup.
W20260806 16:06:48.247613 17967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:48.248415 17967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:48.248531 17967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 16:06:48.251415 18093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.251963 18093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: This master's current role is: LEARNER
W20260806 16:06:48.259253 17967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:48.259392 17967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260806 16:06:48.259132 18112 catalog_manager.cc:1259] Loaded cluster ID: 0b8b98af43d743aa824a44346403a0cc
W20260806 16:06:48.259502 17967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:48.259542 18112 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: loading cluster ID for follower catalog manager: success
I20260806 16:06:48.262889 18112 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: acquiring CA information for follower catalog manager: success
I20260806 16:06:48.265194 18112 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:06:48.270963 17967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab
--principal=oryx/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:40541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=39373
--tserver_master_addrs=127.8.49.62:45969,127.8.49.61:38683,127.8.49.60:39957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:48.273500 17967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:48.275381 17967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:48.291010 18120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:48.291002 18115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:48.291002 18116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:48.291682 17967 server_base.cc:1034] running on GCE node
I20260806 16:06:48.292302 17967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:48.293228 17967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:48.294587 17967 hybrid_clock.cc:648] HybridClock initialized: now 1786032408294552 us; error 83 us; skew 500 ppm
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:48.299932 17967 init.cc:377] Logged in from keytab as oryx/127.8.49.1@KRBTEST.COM (short username oryx)
I20260806 16:06:48.301339 17967 webserver.cc:460] Webserver started at http://127.8.49.1:39373/ using document root <none> and password file <none>
I20260806 16:06:48.302078 17967 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:48.302232 17967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:48.308362 17967 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:48.312057 18129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:48.314112 17967 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:06:48.314369 17967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "702e742c23214b40aa48256127f733f2"
format_stamp: "Formatted at 2026-08-06 16:06:46 on dist-test-slave-7z32"
server_key: "c8970b81bb7f7f542718332847301b55"
server_key_iv: "cdcb877add40c1dcf089acc241ad8fad"
server_key_version: "encryptionkey@0"
I20260806 16:06:48.315318 17967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:48.352851 17967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:48.357326 17967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:48.357585 17967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:48.358314 17967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:48.359404 17967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:48.359485 17967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:48.359562 17967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:48.359607 17967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:48.372591 17967 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:40541
I20260806 16:06:48.372761 18295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:40541 every 8 connection(s)
I20260806 16:06:48.373976 17967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:48.376330 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 17967
I20260806 16:06:48.376629 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 17150
I20260806 16:06:48.394287 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:44165
--local_ip_for_outbound_sockets=127.8.49.2
--tserver_master_addrs=127.8.49.62:45969
--webserver_port=43681
--webserver_interface=127.8.49.2
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab
--principal=oryx/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.49.62:45969,127.8.49.61:38683
--tserver_master_addrs=127.8.49.62:45969,127.8.49.61:38683,127.8.49.60:39957 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.61@KRBTEST.COM
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.60@KRBTEST.COM
I20260806 16:06:48.405866 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.376770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:36517 (local address 127.8.49.62:45969)
0806 16:06:48.380658 (+ 3888us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.380666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.382301 (+ 1635us) server_negotiation.cc:407] Connection header received
0806 16:06:48.384429 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.384437 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.384534 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.384606 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.387518 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.388504 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.390881 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.391289 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.391401 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.398199 (+ 6798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.398209 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.398217 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.398275 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.402912 (+ 4637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.403803 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.403809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.403813 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.403944 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.404324 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.404331 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.404336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.404620 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.404729 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.405448 (+ 719us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.405636 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3710}
I20260806 16:06:48.407704 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.387428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:43709 (local address 127.8.49.60:39957)
0806 16:06:48.387752 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.387757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.387774 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:06:48.387825 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.387827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.387880 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.387979 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.388881 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.390135 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.391586 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.391948 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.392064 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.400099 (+ 8035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.400115 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.400122 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.400253 (+ 131us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.403239 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.404117 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.404123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.404128 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.404242 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.404870 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.404876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.404880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.405479 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.405620 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.407159 (+ 1539us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.407377 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":125,"threads_started":1}
I20260806 16:06:48.408016 18303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.381299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:38683 (local address 127.8.49.1:59701)
0806 16:06:48.382995 (+ 1696us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.383007 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.384095 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.387919 (+ 3824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.387926 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.388325 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.388612 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.388624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.390157 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.390162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.391412 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.391429 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.392232 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.392236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.392756 (+ 520us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.393822 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.393847 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.398131 (+ 4284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.402175 (+ 4044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.402190 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.402230 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.402807 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.403389 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.403405 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.403408 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.403730 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.404472 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.404482 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.405056 (+ 574us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.405359 (+ 303us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.405883 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1621,"mutex_wait_us":497,"spinlock_wait_cycles":38016,"thread_start_us":174,"threads_started":1}
I20260806 16:06:48.408874 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.380628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:39957 (local address 127.8.49.1:43709)
0806 16:06:48.382591 (+ 1963us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.382620 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.384100 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.388136 (+ 4036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.388142 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.388409 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.388616 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.388624 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.390266 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.390269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.391412 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.391431 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.392232 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.392236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.392756 (+ 520us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.393822 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.393852 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.399448 (+ 5596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.403563 (+ 4115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.403569 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.403575 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.403949 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.404376 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.404382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.404386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.404705 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.405696 (+ 991us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.405701 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.405909 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.406619 (+ 710us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.408524 (+ 1905us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1804,"mutex_wait_us":498,"spinlock_wait_cycles":63872,"thread_start_us":196,"threads_started":1}
I20260806 16:06:48.409497 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.387227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:59701 (local address 127.8.49.61:38683)
0806 16:06:48.387396 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.387402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.387422 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:06:48.387487 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.387492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.387567 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.387686 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.388791 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.389972 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.391587 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.391954 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.392054 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.398409 (+ 6355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.398414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.398417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.398462 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.401884 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.403013 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.403019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.403022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.403160 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.403888 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.403895 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.403896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.404281 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.404397 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.409003 (+ 4606us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.409242 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:06:48.411162 18301 heartbeater.cc:347] Connected to a master server at 127.8.49.61:38683
I20260806 16:06:48.411564 18301 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:48.411839 18299 heartbeater.cc:347] Connected to a master server at 127.8.49.60:39957
I20260806 16:06:48.412000 18299 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:48.412516 18301 heartbeater.cc:511] Master 127.8.49.61:38683 requested a full tablet report, sending...
I20260806 16:06:48.413055 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.378095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.1:36517)
0806 16:06:48.381984 (+ 3889us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.382143 (+ 159us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.384049 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.385114 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.385130 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.386192 (+ 1062us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.387254 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.387287 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.388762 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.388769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.390624 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.390643 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.391770 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.391778 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.392153 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.393387 (+ 1234us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.393423 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.397637 (+ 4214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.403060 (+ 5423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.403063 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.403065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.403528 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.404089 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.404093 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.404095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.404193 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.404798 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.404802 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.405056 (+ 254us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.406225 (+ 1169us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.406415 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3520,"mutex_wait_us":32,"spinlock_wait_cycles":17280,"thread_start_us":1002,"threads_started":1}
I20260806 16:06:48.414409 18298 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45969
I20260806 16:06:48.412515 18299 heartbeater.cc:511] Master 127.8.49.60:39957 requested a full tablet report, sending...
I20260806 16:06:48.414670 18298 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:48.415249 18298 heartbeater.cc:511] Master 127.8.49.62:45969 requested a full tablet report, sending...
I20260806 16:06:48.416488 18032 ts_manager.cc:194] Registered new tserver with Master: 702e742c23214b40aa48256127f733f2 (127.8.49.1:40541)
I20260806 16:06:48.417433 16871 ts_manager.cc:194] Re-registered known tserver with Master: 702e742c23214b40aa48256127f733f2 (127.8.49.1:40541)
I20260806 16:06:48.418956 16871 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:36517
I20260806 16:06:48.421247 17758 ts_manager.cc:194] Registered new tserver with Master: 702e742c23214b40aa48256127f733f2 (127.8.49.1:40541)
W20260806 16:06:48.549608 18309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:48.549983 18309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:48.550062 18309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:48.554553 18309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:48.554704 18309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:48.554860 18309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:48.561158 18309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab
--principal=oryx/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:44165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=43681
--tserver_master_addrs=127.8.49.62:45969,127.8.49.61:38683,127.8.49.60:39957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:48.562515 18309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:48.563823 18309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:48.577416 18319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:48.577443 18315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:48.578068 18316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:48.578483 18309 server_base.cc:1034] running on GCE node
I20260806 16:06:48.579370 18309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:48.580710 18309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:48.581987 18309 hybrid_clock.cc:648] HybridClock initialized: now 1786032408581879 us; error 126 us; skew 500 ppm
I20260806 16:06:48.582576 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.574697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.8.49.60:39957)
0806 16:06:48.574981 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.574986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.575413 (+ 427us) server_negotiation.cc:407] Connection header received
0806 16:06:48.575504 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.575508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.575601 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.575727 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:48.576379 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.577450 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.580311 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.581687 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.581921 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.582195 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.582245 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.582378 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:06:48.582576 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.574469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:39957 (local address 127.0.0.1:34774)
0806 16:06:48.575031 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.575052 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.575280 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.575916 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.575922 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.575949 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:48.576098 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.576107 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.577756 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.577761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.579901 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.579932 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.581958 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.581965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.582079 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.582111 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.582205 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.582393 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324}
I20260806 16:06:48.584009 18041 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 16:06:48.584929 18093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62f1112cc8e447e19ce70e0eab5a186b. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:48.585122 18093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: This master's current role is: LEARNER
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:48.586817 18309 init.cc:377] Logged in from keytab as oryx/127.8.49.2@KRBTEST.COM (short username oryx)
I20260806 16:06:48.588084 18309 webserver.cc:460] Webserver started at http://127.8.49.2:43681/ using document root <none> and password file <none>
I20260806 16:06:48.588784 18309 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:48.588886 18309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:48.594233 18309 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:06:48.598278 18329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:48.601075 18309 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 16:06:48.601269 18309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "420507bd250744c2b3a2a83baf2cb501"
format_stamp: "Formatted at 2026-08-06 16:06:46 on dist-test-slave-7z32"
server_key: "07fd207ab7353ff884f066ed953a4e06"
server_key_iv: "028f89309759f1b24bf39c7a725c497e"
server_key_version: "encryptionkey@0"
I20260806 16:06:48.602020 18309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:48.619488 18309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:48.625643 18309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:48.626215 18309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:48.627247 18309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:48.628551 18309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:48.628625 18309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:48.628676 18309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:48.628692 18309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:48.646049 18309 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:44165
I20260806 16:06:48.646330 18490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:44165 every 8 connection(s)
I20260806 16:06:48.647490 18309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:48.651875 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 18309
I20260806 16:06:48.652132 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 17349
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.60@KRBTEST.COM
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.61@KRBTEST.COM
I20260806 16:06:48.673599 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:43059
--local_ip_for_outbound_sockets=127.8.49.3
--tserver_master_addrs=127.8.49.62:45969
--webserver_port=38325
--webserver_interface=127.8.49.3
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab
--principal=oryx/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.49.62:45969,127.8.49.61:38683
--tserver_master_addrs=127.8.49.62:45969,127.8.49.61:38683,127.8.49.60:39957 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
I20260806 16:06:48.684521 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.651391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:53337 (local address 127.8.49.60:39957)
0806 16:06:48.653084 (+ 1693us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.653092 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.655935 (+ 2843us) server_negotiation.cc:407] Connection header received
0806 16:06:48.657868 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.657876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.657979 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.658035 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.661431 (+ 3396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.662727 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.665347 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.665959 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.666260 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.674507 (+ 8247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.674516 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.674523 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.674579 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.678179 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.679808 (+ 1629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.679812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.679814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.680123 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.681171 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.681176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.681178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.682522 (+ 1344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.682641 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.683843 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.684175 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1534}
I20260806 16:06:48.684521 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.650807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:39609 (local address 127.8.49.61:38683)
0806 16:06:48.651200 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.651206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.656746 (+ 5540us) server_negotiation.cc:407] Connection header received
0806 16:06:48.658170 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.658177 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.658249 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.658370 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.661423 (+ 3053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.662709 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.665347 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.665959 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.666117 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.675178 (+ 9061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.675187 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.675194 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.675246 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.678299 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.679804 (+ 1505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.679810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.679813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.679982 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.681171 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.681176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.681178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.681994 (+ 816us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.682123 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.683843 (+ 1720us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.684175 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287}
I20260806 16:06:48.684682 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.650612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:40869 (local address 127.8.49.62:45969)
0806 16:06:48.651061 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.651068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.656029 (+ 4961us) server_negotiation.cc:407] Connection header received
0806 16:06:48.658424 (+ 2395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.658432 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.658533 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.658611 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.661427 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.662517 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.666703 (+ 4186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.667640 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.667933 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.676578 (+ 8645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.676588 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.676594 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.676653 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.680139 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.681198 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.681206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.681211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.681328 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.681918 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.681923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.681925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.682336 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.682588 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.683839 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.684314 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316}
I20260806 16:06:48.687587 18500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.651163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.2:40869)
0806 16:06:48.655569 (+ 4406us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.655695 (+ 126us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.657430 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.658906 (+ 1476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.658913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.659822 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.661106 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.661138 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.662817 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.662865 (+ 48us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.666474 (+ 3609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.666495 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.669606 (+ 3111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.669627 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.669901 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.670689 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.670714 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.675433 (+ 4719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.680394 (+ 4961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.680402 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.680404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.681024 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.681490 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.681497 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.681502 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.681640 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.682617 (+ 977us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.682632 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.683334 (+ 702us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.685094 (+ 1760us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.685701 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4008,"mutex_wait_us":206,"spinlock_wait_cycles":397952,"thread_start_us":257,"threads_started":1}
I20260806 16:06:48.687582 18497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.651161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:39957 (local address 127.8.49.2:53337)
0806 16:06:48.655569 (+ 4408us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.655695 (+ 126us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.657454 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.658714 (+ 1260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.658728 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.659836 (+ 1108us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.661102 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.661138 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.663105 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.663109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.665122 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.665143 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.666533 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.666538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.667342 (+ 804us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.669328 (+ 1986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.669357 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.674032 (+ 4675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.678475 (+ 4443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.678489 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.678525 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.679611 (+ 1086us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.680254 (+ 643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.680259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.680264 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.681004 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.682660 (+ 1656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.682664 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.683513 (+ 849us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.685742 (+ 2229us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.685979 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4282,"mutex_wait_us":818,"spinlock_wait_cycles":143744,"thread_start_us":229,"threads_started":1}
I20260806 16:06:48.687582 18499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.651161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:38683 (local address 127.8.49.2:39609)
0806 16:06:48.656594 (+ 5433us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.656611 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.657980 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.658714 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.658728 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.659838 (+ 1110us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.661102 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.661138 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.663105 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.663109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.665122 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.665143 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.666534 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.666538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.667342 (+ 804us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.669738 (+ 2396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.669754 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.674876 (+ 5122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.678475 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.678489 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.678525 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.679611 (+ 1086us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.680575 (+ 964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.680579 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.680581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.681004 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.682701 (+ 1697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.682704 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.683513 (+ 809us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.684972 (+ 1459us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.686061 (+ 1089us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5327,"mutex_wait_us":2811,"spinlock_wait_cycles":92416,"thread_start_us":2113,"threads_started":1}
I20260806 16:06:48.689836 18493 heartbeater.cc:347] Connected to a master server at 127.8.49.60:39957
I20260806 16:06:48.690230 18493 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:48.689863 18492 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45969
I20260806 16:06:48.689865 18494 heartbeater.cc:347] Connected to a master server at 127.8.49.61:38683
I20260806 16:06:48.691159 18492 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:48.691133 18494 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:48.691138 18493 heartbeater.cc:511] Master 127.8.49.60:39957 requested a full tablet report, sending...
I20260806 16:06:48.691391 18494 heartbeater.cc:511] Master 127.8.49.61:38683 requested a full tablet report, sending...
I20260806 16:06:48.691550 18492 heartbeater.cc:511] Master 127.8.49.62:45969 requested a full tablet report, sending...
I20260806 16:06:48.693048 16871 ts_manager.cc:194] Re-registered known tserver with Master: 420507bd250744c2b3a2a83baf2cb501 (127.8.49.2:44165)
I20260806 16:06:48.693117 17758 ts_manager.cc:194] Registered new tserver with Master: 420507bd250744c2b3a2a83baf2cb501 (127.8.49.2:44165)
I20260806 16:06:48.693117 18032 ts_manager.cc:194] Registered new tserver with Master: 420507bd250744c2b3a2a83baf2cb501 (127.8.49.2:44165)
I20260806 16:06:48.693951 16871 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:40869
W20260806 16:06:48.850556 18502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:48.851114 18502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:48.851356 18502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:48.857928 18502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:48.858057 18502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:48.858168 18502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:48.867018 18502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab
--principal=oryx/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:43059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=38325
--tserver_master_addrs=127.8.49.62:45969,127.8.49.61:38683,127.8.49.60:39957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:48.868604 18502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:48.869908 18502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:48.887768 18510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:48.887768 18508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:48.887776 18513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:48.888357 18502 server_base.cc:1034] running on GCE node
I20260806 16:06:48.889034 18502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:48.889945 18502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:48.893456 18502 hybrid_clock.cc:648] HybridClock initialized: now 1786032408893306 us; error 160 us; skew 500 ppm
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:48.898082 18502 init.cc:377] Logged in from keytab as oryx/127.8.49.3@KRBTEST.COM (short username oryx)
I20260806 16:06:48.899588 18502 webserver.cc:460] Webserver started at http://127.8.49.3:38325/ using document root <none> and password file <none>
I20260806 16:06:48.900377 18502 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:48.900542 18502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:48.907407 18502 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:06:48.911814 18526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:48.914153 18502 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:06:48.914408 18502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "2c3dc20b5f304e69a6cf42f8baa83e17"
format_stamp: "Formatted at 2026-08-06 16:06:47 on dist-test-slave-7z32"
server_key: "7a2ea1c5d6700457185004629252ea85"
server_key_iv: "004651382b7ddde70aed8d9ba0011465"
server_key_version: "encryptionkey@0"
I20260806 16:06:48.915328 18502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:48.941566 18502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:48.946722 18502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:48.947060 18502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:48.947871 18502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:06:48.949150 18502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:48.949226 18502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:48.949309 18502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:48.949362 18502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:48.964978 18502 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:43059
I20260806 16:06:48.965059 18690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:43059 every 8 connection(s)
I20260806 16:06:48.966642 18502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:48.971000 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 18502
I20260806 16:06:48.988766 18713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.974021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49022 (local address 127.8.49.62:45969)
0806 16:06:48.974334 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.974339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.975259 (+ 920us) server_negotiation.cc:407] Connection header received
0806 16:06:48.975491 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.975495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.975563 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.975634 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.977148 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.978044 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.979696 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.980043 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.980133 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.982552 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.982557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.982560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.982600 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.985849 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.986740 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.986745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.986747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.986945 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.987357 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.987361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.987363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.987692 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.987835 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.988301 (+ 466us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.988519 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":131,"threads_started":1}
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.61@KRBTEST.COM
Aug 06 16:06:48 dist-test-slave-7z32 krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032408, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.60@KRBTEST.COM
I20260806 16:06:48.995564 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.974309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34780 (local address 127.8.49.60:39957)
0806 16:06:48.974685 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.974690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.974871 (+ 181us) server_negotiation.cc:407] Connection header received
0806 16:06:48.975045 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.975051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.975123 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.975899 (+ 776us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.976820 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.977635 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.979216 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.979562 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.979685 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.982680 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.982685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.982689 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.982731 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.986009 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.987841 (+ 1832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.987854 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.987858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.988010 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.988445 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.988449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.988451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.988724 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.994905 (+ 6181us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.995052 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.995307 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":125,"threads_started":1}
I20260806 16:06:48.998257 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.971730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:39163 (local address 127.8.49.62:45969)
0806 16:06:48.971918 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.971925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.978865 (+ 6940us) server_negotiation.cc:407] Connection header received
0806 16:06:48.979257 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.979261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.979329 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.979386 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.983057 (+ 3671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.983827 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.987392 (+ 3565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.987793 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.987901 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.992144 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.992152 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.992158 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.992194 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.994583 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.995511 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.995516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.995518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.995620 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.996133 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.996137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.996139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.996423 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.996522 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.997740 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.998063 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 16:06:48.998482 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.969618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:50761 (local address 127.8.49.61:38683)
0806 16:06:48.969845 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.969850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.978535 (+ 8685us) server_negotiation.cc:407] Connection header received
0806 16:06:48.979552 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.979556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.979632 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.979688 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.981967 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.984452 (+ 2485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.988384 (+ 3932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.988773 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.988887 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.993080 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.993088 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.993094 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.993137 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.995234 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.995890 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.995895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.995897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.996143 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:48.996637 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:48.996641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:48.996644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:48.997075 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:48.997221 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:48.997981 (+ 760us) server_negotiation.cc:299] Negotiation successful
0806 16:06:48.998138 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":108}
I20260806 16:06:48.994498 16871 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 16:06:49.000717 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.970185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.3:39163)
0806 16:06:48.978736 (+ 8551us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.978750 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.978992 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.979817 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.979823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.980137 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.981807 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.981817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.984680 (+ 2863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.984687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.987206 (+ 2519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.987227 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.988178 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.988184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.988485 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.989226 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.989261 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.991855 (+ 2594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.994773 (+ 2918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.994783 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.994808 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.995354 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.995750 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.995754 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.995756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.996001 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.996607 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.996616 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.997373 (+ 757us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.998597 (+ 1224us) client_negotiation.cc:240] Negotiation successful
0806 16:06:48.999255 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8450,"mutex_wait_us":331,"thread_start_us":178,"threads_started":1}
W20260806 16:06:49.000677 16871 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:49.003202 18692 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45969
I20260806 16:06:49.003674 18692 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:49.004787 18692 heartbeater.cc:511] Master 127.8.49.62:45969 requested a full tablet report, sending...
I20260806 16:06:49.004875 18712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.973886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.8.49.61:38683)
0806 16:06:48.975213 (+ 1327us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.975220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.975238 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:06:48.975298 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.975301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.975353 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.975448 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.976681 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.977559 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.978857 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.979567 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.979680 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.982552 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:48.982557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:48.982560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:48.982600 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:48.999536 (+ 16936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.000414 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.000420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.000424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.000559 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.001449 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.001455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.001458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.001881 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:49.002183 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.003599 (+ 1416us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.004599 (+ 1000us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1229,"thread_start_us":91,"threads_started":1}
I20260806 16:06:49.006206 17969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: NON_VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: true } } }
I20260806 16:06:49.006395 17969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:49.006453 16868 ts_manager.cc:194] Re-registered known tserver with Master: 2c3dc20b5f304e69a6cf42f8baa83e17 (127.8.49.3:43059)
I20260806 16:06:49.007311 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.970062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:42621 (local address 127.8.49.60:39957)
0806 16:06:48.970189 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:48.970193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:48.972629 (+ 2436us) server_negotiation.cc:407] Connection header received
0806 16:06:48.975502 (+ 2873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:48.975507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:48.975566 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:48.975623 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:48.985338 (+ 9715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.986438 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.990048 (+ 3610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.990418 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.990559 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.001502 (+ 10943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:49.001510 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:49.001513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:49.001556 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:49.004596 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.005310 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.005315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.005317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.005441 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.005896 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.005900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.005901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.006193 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:49.006373 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.006937 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.007122 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 16:06:49.007630 16868 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:39163
I20260806 16:06:49.008800 18698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.970184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:39957 (local address 127.8.49.3:42621)
0806 16:06:48.972383 (+ 2199us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.972467 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.975175 (+ 2708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.976242 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.976253 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.976971 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.977962 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.978026 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.987658 (+ 9632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.987661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.989874 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.989892 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.990662 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.990667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.990782 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.992983 (+ 2201us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.993003 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.998111 (+ 5108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:49.004810 (+ 6699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:49.004814 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:49.004817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:49.005163 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:49.005578 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:49.005581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:49.005584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:49.005763 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:49.006444 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:49.006447 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:49.006703 (+ 256us) client_negotiation.cc:769] Sending connection context
0806 16:06:49.007390 (+ 687us) client_negotiation.cc:240] Negotiation successful
0806 16:06:49.007974 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1844,"mutex_wait_us":1334,"thread_start_us":216,"threads_started":1}
I20260806 16:06:49.010140 18694 heartbeater.cc:347] Connected to a master server at 127.8.49.60:39957
I20260806 16:06:49.010345 18694 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:49.010643 18694 heartbeater.cc:511] Master 127.8.49.60:39957 requested a full tablet report, sending...
I20260806 16:06:49.013835 18032 ts_manager.cc:194] Registered new tserver with Master: 2c3dc20b5f304e69a6cf42f8baa83e17 (127.8.49.3:43059)
I20260806 16:06:49.015498 18701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:48.970184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:38683 (local address 127.8.49.3:50761)
0806 16:06:48.978418 (+ 8234us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:48.978437 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:48.979107 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:48.981200 (+ 2093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:48.981204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:48.981451 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:48.981642 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.981650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.984679 (+ 3029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.984684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.987459 (+ 2775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:48.987478 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:48.990114 (+ 2636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:48.990118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:48.990251 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:48.990933 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:48.990952 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:48.992814 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:48.995410 (+ 2596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.995414 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.995417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.995762 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.996361 (+ 599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:48.996365 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:48.996367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:48.996492 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:48.997437 (+ 945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:48.997441 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:48.997574 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 16:06:48.998712 (+ 1138us) client_negotiation.cc:240] Negotiation successful
0806 16:06:49.015226 (+ 16514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8059,"mutex_wait_us":17363,"spinlock_wait_cycles":56704,"thread_start_us":182,"threads_started":1}
I20260806 16:06:49.016587 18719 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b: attempting to promote NON_VOTER 939100169d094eeba3e04bb2ea5304ff to VOTER
I20260806 16:06:49.017148 18719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } }
I20260806 16:06:49.017407 18695 heartbeater.cc:347] Connected to a master server at 127.8.49.61:38683
I20260806 16:06:49.017575 18695 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:49.017957 18695 heartbeater.cc:511] Master 127.8.49.61:38683 requested a full tablet report, sending...
I20260806 16:06:49.019260 17758 ts_manager.cc:194] Registered new tserver with Master: 2c3dc20b5f304e69a6cf42f8baa83e17 (127.8.49.3:43059)
I20260806 16:06:49.021778 18041 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 LEARNER]: Refusing update from remote peer 62f1112cc8e447e19ce70e0eab5a186b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 16:06:49.021778 17771 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Refusing update from remote peer 62f1112cc8e447e19ce70e0eab5a186b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 16:06:49.022583 18731 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 16:06:49.022969 18719 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 16:06:49.027451 17969 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 939100169d094eeba3e04bb2ea5304ff (127.8.49.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:49.028043 17771 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 939100169d094eeba3e04bb2ea5304ff (127.8.49.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:49.029738 17969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:49.029925 17969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:49.029887 18734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:49.030071 18734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:06:49.030113 17969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:49.030220 17969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:49.033195 18731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:49.033394 18731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:49.034020 18041 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 939100169d094eeba3e04bb2ea5304ff (127.8.49.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:49.037627 18093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62f1112cc8e447e19ce70e0eab5a186b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:49.037809 18093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:06:49.044373 18740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.035051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:44165 (local address 127.0.0.1:46648)
0806 16:06:49.035415 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:49.035435 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:49.035552 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:49.036987 (+ 1435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:49.036992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:49.037018 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:49.037423 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:49.037437 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.039889 (+ 2452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.039901 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:49.041782 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:49.041797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.043849 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.043854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:49.043968 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.043994 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:06:49.044053 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 16:06:49.044151 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":157,"threads_started":1}
I20260806 16:06:49.044983 18736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.034275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:43059 (local address 127.0.0.1:59328)
0806 16:06:49.034696 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:49.034717 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:49.034887 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:49.038598 (+ 3711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:49.038603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:49.038636 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:49.038762 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:49.038769 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.040731 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.040735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:49.042336 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:49.042348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.043849 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.043854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:49.044667 (+ 813us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.044696 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 16:06:49.044777 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 16:06:49.044862 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":708,"thread_start_us":220,"threads_started":1}
I20260806 16:06:49.045671 18742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.035599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46648 (local address 127.8.49.2:44165)
0806 16:06:49.036193 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:49.036213 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:49.036247 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:06:49.036372 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:49.036380 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:49.036764 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:49.037762 (+ 998us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:49.037986 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.039434 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.042088 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.043665 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.045234 (+ 1569us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.045392 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.045455 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.045553 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":133,"threads_started":1}
I20260806 16:06:49.047358 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.035155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59328 (local address 127.8.49.3:43059)
0806 16:06:49.037422 (+ 2267us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:49.037442 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:49.037486 (+ 44us) server_negotiation.cc:407] Connection header received
0806 16:06:49.037596 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:49.037603 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:49.038320 (+ 717us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:49.038558 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:49.039064 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.040544 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.042547 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.043665 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.043903 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.044107 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.047026 (+ 2919us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.047186 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1966,"thread_start_us":212,"threads_started":1}
I20260806 16:06:49.049098 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.033522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40541 (local address 127.0.0.1:57218)
0806 16:06:49.034044 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:49.034064 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:49.034178 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:49.039586 (+ 5408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:49.039590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:49.039615 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:49.039710 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:49.039716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.042242 (+ 2526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.042245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:49.044324 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:49.044343 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.048617 (+ 4274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.048623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:49.048733 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.048757 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:06:49.048852 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 16:06:49.048951 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409}
I20260806 16:06:49.049268 18737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.034377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.8.49.1:40541)
0806 16:06:49.039005 (+ 4628us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:49.039013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:49.039045 (+ 32us) server_negotiation.cc:407] Connection header received
0806 16:06:49.039143 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:49.039150 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:49.039391 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:49.039580 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:49.040200 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.042075 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.044633 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.047491 (+ 2858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.048636 (+ 1145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.048878 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.048940 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.049055 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4471,"thread_start_us":163,"threads_started":1}
I20260806 16:06:49.050019 18395 tablet_service.cc:1467] Processing CreateTablet for tablet 539c96c14b524e50887339e6d7923276 (DEFAULT_TABLE table=test-table [id=1c7a4404c26f4f00958746dd83a7080b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:49.050863 18596 tablet_service.cc:1467] Processing CreateTablet for tablet 539c96c14b524e50887339e6d7923276 (DEFAULT_TABLE table=test-table [id=1c7a4404c26f4f00958746dd83a7080b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:49.051672 18395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 539c96c14b524e50887339e6d7923276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:49.052067 18596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 539c96c14b524e50887339e6d7923276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:49.053412 18197 tablet_service.cc:1467] Processing CreateTablet for tablet 539c96c14b524e50887339e6d7923276 (DEFAULT_TABLE table=test-table [id=1c7a4404c26f4f00958746dd83a7080b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:49.055114 18197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 539c96c14b524e50887339e6d7923276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:49.063323 18743 tablet_bootstrap.cc:492] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: Bootstrap starting.
I20260806 16:06:49.064607 18745 tablet_bootstrap.cc:492] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: Bootstrap starting.
I20260806 16:06:49.065842 18743 tablet_bootstrap.cc:654] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:49.066277 18746 tablet_bootstrap.cc:492] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: Bootstrap starting.
I20260806 16:06:49.066956 18743 log.cc:824] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:49.068071 18745 tablet_bootstrap.cc:654] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:49.069703 18746 tablet_bootstrap.cc:654] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:49.069861 18745 log.cc:824] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:49.070798 18746 log.cc:824] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:49.073794 18743 tablet_bootstrap.cc:492] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: No bootstrap required, opened a new log
I20260806 16:06:49.073786 18746 tablet_bootstrap.cc:492] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: No bootstrap required, opened a new log
I20260806 16:06:49.073987 18745 tablet_bootstrap.cc:492] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: No bootstrap required, opened a new log
I20260806 16:06:49.074103 18743 ts_tablet_manager.cc:1397] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260806 16:06:49.074103 18746 ts_tablet_manager.cc:1397] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 16:06:49.074314 18745 ts_tablet_manager.cc:1397] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 16:06:49.079144 18745 raft_consensus.cc:348] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } }
I20260806 16:06:49.079520 18745 raft_consensus.cc:374] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:49.079588 18745 raft_consensus.cc:729] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3dc20b5f304e69a6cf42f8baa83e17, State: Initialized, Role: FOLLOWER
I20260806 16:06:49.080176 18745 consensus_queue.cc:260] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } }
I20260806 16:06:49.081097 18692 heartbeater.cc:503] Master 127.8.49.62:45969 was elected leader, sending a full tablet report...
I20260806 16:06:49.080876 18746 raft_consensus.cc:348] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } }
I20260806 16:06:49.081372 18746 raft_consensus.cc:374] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:49.081439 18746 raft_consensus.cc:729] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 702e742c23214b40aa48256127f733f2, State: Initialized, Role: FOLLOWER
I20260806 16:06:49.081667 18745 ts_tablet_manager.cc:1428] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:06:49.082525 18746 consensus_queue.cc:260] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } }
I20260806 16:06:49.084671 18298 heartbeater.cc:503] Master 127.8.49.62:45969 was elected leader, sending a full tablet report...
I20260806 16:06:49.085486 18746 ts_tablet_manager.cc:1428] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260806 16:06:49.089452 18743 raft_consensus.cc:348] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } }
I20260806 16:06:49.089922 18743 raft_consensus.cc:374] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:49.090018 18743 raft_consensus.cc:729] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 420507bd250744c2b3a2a83baf2cb501, State: Initialized, Role: FOLLOWER
I20260806 16:06:49.090660 18743 consensus_queue.cc:260] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } }
I20260806 16:06:49.091820 18492 heartbeater.cc:503] Master 127.8.49.62:45969 was elected leader, sending a full tablet report...
I20260806 16:06:49.092087 18743 ts_tablet_manager.cc:1428] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: Time spent starting tablet: real 0.018s user 0.006s sys 0.004s
W20260806 16:06:49.126564 18302 tablet.cc:2369] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:06:49.150903 18495 tablet.cc:2369] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:06:49.219720 18696 tablet.cc:2369] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:49.341003 18753 raft_consensus.cc:482] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:49.341284 18753 raft_consensus.cc:504] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } }
I20260806 16:06:49.343052 18753 leader_election.cc:288] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 702e742c23214b40aa48256127f733f2 (127.8.49.1:40541), 420507bd250744c2b3a2a83baf2cb501 (127.8.49.2:44165)
I20260806 16:06:49.348707 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.343543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40541 (local address 127.8.49.3:47843)
0806 16:06:49.343820 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:49.343844 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:49.344115 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:49.344545 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:49.344548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:49.344570 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:49.344944 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:49.344955 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.345937 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.345941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:49.347147 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:49.347161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.348039 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.348044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:49.348177 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.348216 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 16:06:49.348329 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 16:06:49.348547 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 16:06:49.348831 18737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.343741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:47843 (local address 127.8.49.1:40541)
0806 16:06:49.343937 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:49.343943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:49.344012 (+ 69us) server_negotiation.cc:407] Connection header received
0806 16:06:49.344331 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:49.344335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:49.344408 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:49.344493 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:49.345144 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.345774 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.347314 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.347821 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.348029 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.348115 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.348466 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.348588 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 16:06:49.349054 18701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.344024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:44165 (local address 127.8.49.3:47523)
0806 16:06:49.344137 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:49.344156 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:49.344233 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:49.344917 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:49.344920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:49.344947 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:49.345117 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:49.345127 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.346199 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.346202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:49.347218 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:49.347231 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.348489 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.348493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:49.348629 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.348670 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 16:06:49.348769 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 16:06:49.348880 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 16:06:49.349166 18742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.344204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:47523 (local address 127.8.49.2:44165)
0806 16:06:49.344400 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:49.344405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:49.344420 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:06:49.344510 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:49.344516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:49.344582 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:49.344903 (+ 321us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:49.345315 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.345990 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.347492 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.348173 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.348366 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.348458 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.348867 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.348993 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 16:06:49.350126 18231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "539c96c14b524e50887339e6d7923276" candidate_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "702e742c23214b40aa48256127f733f2" is_pre_election: true
I20260806 16:06:49.350178 18424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "539c96c14b524e50887339e6d7923276" candidate_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "420507bd250744c2b3a2a83baf2cb501" is_pre_election: true
I20260806 16:06:49.350768 18424 raft_consensus.cc:2454] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c3dc20b5f304e69a6cf42f8baa83e17 in term 0.
I20260806 16:06:49.350792 18231 raft_consensus.cc:2454] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c3dc20b5f304e69a6cf42f8baa83e17 in term 0.
I20260806 16:06:49.351768 18528 leader_election.cc:302] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3dc20b5f304e69a6cf42f8baa83e17, 420507bd250744c2b3a2a83baf2cb501; no voters:
I20260806 16:06:49.352355 18753 raft_consensus.cc:2781] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:49.352583 18753 raft_consensus.cc:482] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:49.352684 18753 raft_consensus.cc:3037] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:49.354007 18753 raft_consensus.cc:504] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } }
I20260806 16:06:49.355379 18753 leader_election.cc:288] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [CANDIDATE]: Term 1 election: Requested vote from peers 702e742c23214b40aa48256127f733f2 (127.8.49.1:40541), 420507bd250744c2b3a2a83baf2cb501 (127.8.49.2:44165)
I20260806 16:06:49.355695 18231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "539c96c14b524e50887339e6d7923276" candidate_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "702e742c23214b40aa48256127f733f2"
I20260806 16:06:49.355911 18231 raft_consensus.cc:3037] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:49.356612 18424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "539c96c14b524e50887339e6d7923276" candidate_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "420507bd250744c2b3a2a83baf2cb501"
I20260806 16:06:49.356930 18424 raft_consensus.cc:3037] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:49.357645 18231 raft_consensus.cc:2454] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c3dc20b5f304e69a6cf42f8baa83e17 in term 1.
I20260806 16:06:49.358279 18528 leader_election.cc:302] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3dc20b5f304e69a6cf42f8baa83e17, 702e742c23214b40aa48256127f733f2; no voters:
I20260806 16:06:49.358575 18753 raft_consensus.cc:2781] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:49.358925 18753 raft_consensus.cc:686] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 1 LEADER]: Becoming Leader. State: Replica: 2c3dc20b5f304e69a6cf42f8baa83e17, State: Running, Role: LEADER
I20260806 16:06:49.359159 18424 raft_consensus.cc:2454] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c3dc20b5f304e69a6cf42f8baa83e17 in term 1.
I20260806 16:06:49.359431 18753 consensus_queue.cc:237] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } }
I20260806 16:06:49.364588 16868 catalog_manager.cc:5495] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c3dc20b5f304e69a6cf42f8baa83e17 (127.8.49.3). New cstate: current_term: 1 leader_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } health_report { overall_health: UNKNOWN } } }
I20260806 16:06:49.424830 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.418719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.8.49.3:43059)
0806 16:06:49.419089 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:49.419094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:49.419114 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:06:49.419231 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:49.419233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:49.419342 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:49.419679 (+ 337us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:49.420264 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.420989 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.422127 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.422407 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.422545 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.423086 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:49.424096 (+ 1010us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:49.424383 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.424496 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.424597 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 16:06:49.434559 18424 raft_consensus.cc:1261] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501 [term 1 FOLLOWER]: Refusing update from remote peer 2c3dc20b5f304e69a6cf42f8baa83e17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:49.434559 18231 raft_consensus.cc:1261] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2 [term 1 FOLLOWER]: Refusing update from remote peer 2c3dc20b5f304e69a6cf42f8baa83e17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:49.435917 18756 consensus_queue.cc:1035] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:49.436393 18753 consensus_queue.cc:1035] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:49.461818 18761 mvcc.cc:205] Tried to move back new op lower bound from 7315588749031788544 to 7315588748740190208. Current Snapshot: MvccSnapshot[applied={T|T < 7315588749031788544}]
I20260806 16:06:49.472299 18762 mvcc.cc:205] Tried to move back new op lower bound from 7315588749031788544 to 7315588748740190208. Current Snapshot: MvccSnapshot[applied={T|T < 7315588749031788544 or (T in {7315588749031788544})}]
I20260806 16:06:49.474895 16871 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:49.475178 16871 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:49.491840 16871 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 62f1112cc8e447e19ce70e0eab5a186b: Sending DeleteTablet for 3 replicas of tablet 539c96c14b524e50887339e6d7923276
I20260806 16:06:49.493376 18197 tablet_service.cc:1514] Processing DeleteTablet for tablet 539c96c14b524e50887339e6d7923276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:49 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:57218
I20260806 16:06:49.493651 18596 tablet_service.cc:1514] Processing DeleteTablet for tablet 539c96c14b524e50887339e6d7923276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:49 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:59328
I20260806 16:06:49.495581 18395 tablet_service.cc:1514] Processing DeleteTablet for tablet 539c96c14b524e50887339e6d7923276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:49 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:46648
I20260806 16:06:49.499579 18774 tablet_replica.cc:330] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: stopping tablet replica
I20260806 16:06:49.500160 18774 raft_consensus.cc:2229] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:49.500658 18773 tablet_replica.cc:330] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: stopping tablet replica
I20260806 16:06:49.501017 18773 raft_consensus.cc:2229] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:49.501269 18774 raft_consensus.cc:2258] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:49.501430 18773 raft_consensus.cc:2258] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:49.502928 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 16823
I20260806 16:06:49.506706 18774 ts_tablet_manager.cc:1905] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:49.508534 18773 ts_tablet_manager.cc:1905] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:49.512068 18774 ts_tablet_manager.cc:1918] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:49.512648 18774 log.cc:1196] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/539c96c14b524e50887339e6d7923276
I20260806 16:06:49.513299 18774 ts_tablet_manager.cc:1939] T 539c96c14b524e50887339e6d7923276 P 2c3dc20b5f304e69a6cf42f8baa83e17: Deleting consensus metadata
I20260806 16:06:49.523341 18773 ts_tablet_manager.cc:1918] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:49.523648 18773 log.cc:1196] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-0/wal/wals/539c96c14b524e50887339e6d7923276
I20260806 16:06:49.524320 18773 ts_tablet_manager.cc:1939] T 539c96c14b524e50887339e6d7923276 P 702e742c23214b40aa48256127f733f2: Deleting consensus metadata
I20260806 16:06:49.535126 18777 tablet_replica.cc:330] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: stopping tablet replica
I20260806 16:06:49.535715 18777 raft_consensus.cc:2229] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:49.536415 18777 raft_consensus.cc:2258] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 16:06:49.537019 18531 connection.cc:535] server connection from 127.0.0.1:59328 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 16:06:49.539330 18130 connection.cc:535] server connection from 127.0.0.1:57218 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 16:06:49.542215 18777 ts_tablet_manager.cc:1905] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:49.549505 18712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.500697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41386 (local address 127.8.49.61:38683)
0806 16:06:49.500895 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:49.500900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:49.531447 (+ 30547us) server_negotiation.cc:407] Connection header received
0806 16:06:49.531763 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:49.531770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:49.531863 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:49.531990 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:49.535634 (+ 3644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.536754 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.538875 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.539530 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.539655 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.543947 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:49.543955 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:49.543962 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:49.544022 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:49.546649 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.547500 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.547507 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.547511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.547630 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.548014 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.548019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.548023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.548338 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:49.548524 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.549019 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.549215 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 16:06:49.550769 18777 ts_tablet_manager.cc:1918] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:49.551105 18777 log.cc:1196] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/539c96c14b524e50887339e6d7923276
I20260806 16:06:49.551604 18777 ts_tablet_manager.cc:1939] T 539c96c14b524e50887339e6d7923276 P 420507bd250744c2b3a2a83baf2cb501: Deleting consensus metadata
W20260806 16:06:49.553597 18333 connection.cc:441] server connection from 127.0.0.1:46648 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46648 (request call id 1) could send its response
I20260806 16:06:49.565860 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.499603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34792 (local address 127.8.49.60:39957)
0806 16:06:49.499760 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:49.499766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:49.552266 (+ 52500us) server_negotiation.cc:407] Connection header received
0806 16:06:49.552548 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:49.552554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:49.552623 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:49.552681 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:49.553936 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.554782 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.556384 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.556813 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.556912 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.559445 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:49.559450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:49.559453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:49.559489 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:49.562036 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.563180 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.563186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.563188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.563326 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.563710 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.563713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.563714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.564395 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:49.564533 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.565460 (+ 927us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.565629 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 16:06:49.574892 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.554538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.8.49.60:39957)
0806 16:06:49.555246 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:49.555252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:49.557099 (+ 1847us) server_negotiation.cc:407] Connection header received
0806 16:06:49.557348 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:49.557353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:49.557425 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:49.557506 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:49.559904 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.560854 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.562335 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.562879 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.562986 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.568238 (+ 5252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:49.568244 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:49.568248 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:49.568287 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:49.571046 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.572050 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.572056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.572058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.572210 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.573005 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.573012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.573016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.573373 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:49.573555 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.574346 (+ 791us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.574637 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":96,"threads_started":1}
I20260806 16:06:49.583770 18712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:49.555773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.8.49.61:38683)
0806 16:06:49.555938 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:49.555943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:49.565703 (+ 9760us) server_negotiation.cc:407] Connection header received
0806 16:06:49.565952 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:49.565956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:49.566021 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:49.566085 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:49.569018 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.569979 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.572988 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:49.573430 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:49.573566 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:49.577137 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:49.577143 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:49.577146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:49.577192 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:49.579590 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.580929 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.580935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.580937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.581100 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:49.581657 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:49.581661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:49.581663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:49.582079 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:49.582436 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:49.583216 (+ 780us) server_negotiation.cc:299] Negotiation successful
0806 16:06:49.583461 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 16:06:50.461335 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:50.460593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.2:46017)
0806 16:06:50.460983 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:50.461187 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":127,"threads_started":1}
W20260806 16:06:50.461839 18492 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:45969 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
I20260806 16:06:50.465329 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:50.464731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.1:33057)
0806 16:06:50.465109 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:50.465221 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":100,"threads_started":1}
W20260806 16:06:50.465996 18298 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:45969 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
I20260806 16:06:50.472612 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:50.471881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.3:32951)
0806 16:06:50.472338 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:50.472458 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":117,"threads_started":1}
W20260806 16:06:50.473170 18692 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:45969 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
I20260806 16:06:50.989025 18797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 62f1112cc8e447e19ce70e0eab5a186b)
I20260806 16:06:50.989616 18797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } }
I20260806 16:06:50.990139 18798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 62f1112cc8e447e19ce70e0eab5a186b)
I20260806 16:06:50.990310 18798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } }
I20260806 16:06:50.991201 18798 leader_election.cc:288] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969), 6274597ce8be486fba4a0613d099ea9f (127.8.49.61:38683)
I20260806 16:06:50.992074 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:50.991414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49502)
0806 16:06:50.991811 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:50.991943 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":76,"threads_started":1}
W20260806 16:06:50.992314 18005 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
I20260806 16:06:50.992434 18797 leader_election.cc:288] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969), 939100169d094eeba3e04bb2ea5304ff (127.8.49.60:39957)
I20260806 16:06:50.993117 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:50.991625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49508)
0806 16:06:50.992883 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:50.992987 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":104,"threads_started":1}
I20260806 16:06:50.993422 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:50.993118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49518)
0806 16:06:50.993271 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:50.993339 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 16:06:50.993774 18005 leader_election.cc:334] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969): Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
W20260806 16:06:50.993860 17734 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
I20260806 16:06:50.995260 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:50.994896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49524)
0806 16:06:50.995060 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:50.995133 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260806 16:06:50.995435 17734 leader_election.cc:334] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969): Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
I20260806 16:06:50.998263 18805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:50.991785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41400 (local address 127.8.49.61:38683)
0806 16:06:50.992173 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:50.992180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:50.992832 (+ 652us) server_negotiation.cc:407] Connection header received
0806 16:06:50.993691 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:50.993696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:50.993755 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:50.993848 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:50.994231 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:50.995077 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:50.996774 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:50.997414 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:50.997598 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:50.997663 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:50.997995 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 16:06:50.998100 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":114,"threads_started":1}
I20260806 16:06:50.998263 18808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:50.992785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34804 (local address 127.8.49.60:39957)
0806 16:06:50.993086 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:50.993096 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:50.993112 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:06:50.993173 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:50.993176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:50.993236 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:50.993385 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:50.993761 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:50.994734 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:50.996593 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:50.997347 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:50.997495 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:50.997576 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:50.997995 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 16:06:50.998117 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":118,"threads_started":1}
I20260806 16:06:50.998401 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:50.991642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:38683 (local address 127.0.0.1:41400)
0806 16:06:50.992704 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:50.992726 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:50.993380 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:50.993995 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:50.993998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:50.994017 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:50.994101 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:50.994106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:50.995220 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:50.995224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:50.996620 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:50.996630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:50.997696 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:50.997701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:50.997797 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:50.997822 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 16:06:50.997912 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 16:06:50.998022 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":110,"threads_started":1}
I20260806 16:06:50.998383 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:50.992243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:39957 (local address 127.0.0.1:34804)
0806 16:06:50.992547 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:50.992567 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:50.992722 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:50.993407 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:50.993411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:50.993435 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:50.993571 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:50.993577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:50.994921 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:50.994926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:50.996404 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:50.996418 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:50.997651 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:50.997656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:50.997768 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:50.997806 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 16:06:50.997912 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 16:06:50.998022 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":68,"threads_started":1}
I20260806 16:06:50.999511 18041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6274597ce8be486fba4a0613d099ea9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "939100169d094eeba3e04bb2ea5304ff" is_pre_election: true
I20260806 16:06:50.999629 17771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "939100169d094eeba3e04bb2ea5304ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6274597ce8be486fba4a0613d099ea9f" is_pre_election: true
I20260806 16:06:50.999851 17771 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939100169d094eeba3e04bb2ea5304ff in term 1.
I20260806 16:06:50.999872 18041 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6274597ce8be486fba4a0613d099ea9f in term 1.
I20260806 16:06:51.000289 18006 leader_election.cc:302] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6274597ce8be486fba4a0613d099ea9f, 939100169d094eeba3e04bb2ea5304ff; no voters: 62f1112cc8e447e19ce70e0eab5a186b
I20260806 16:06:51.000337 17733 leader_election.cc:302] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6274597ce8be486fba4a0613d099ea9f, 939100169d094eeba3e04bb2ea5304ff; no voters: 62f1112cc8e447e19ce70e0eab5a186b
I20260806 16:06:51.000671 18798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 16:06:51.000754 18798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Starting leader election (detected failure of leader 62f1112cc8e447e19ce70e0eab5a186b)
I20260806 16:06:51.000659 18797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 16:06:51.000806 18798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:06:51.000826 18797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 62f1112cc8e447e19ce70e0eab5a186b)
I20260806 16:06:51.000905 18797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:06:51.002144 18798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } }
I20260806 16:06:51.002439 18797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } }
I20260806 16:06:51.002646 18798 leader_election.cc:288] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 2 election: Requested vote from peers 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969), 6274597ce8be486fba4a0613d099ea9f (127.8.49.61:38683)
I20260806 16:06:51.003185 18797 leader_election.cc:288] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [CANDIDATE]: Term 2 election: Requested vote from peers 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969), 939100169d094eeba3e04bb2ea5304ff (127.8.49.60:39957)
I20260806 16:06:51.003299 17771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "939100169d094eeba3e04bb2ea5304ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6274597ce8be486fba4a0613d099ea9f"
I20260806 16:06:51.003386 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:51.002816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49538)
0806 16:06:51.003021 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:51.003085 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 16:06:51.003485 17771 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 939100169d094eeba3e04bb2ea5304ff in current term 2: Already voted for candidate 6274597ce8be486fba4a0613d099ea9f in this term.
I20260806 16:06:51.003679 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:51.003352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49540)
0806 16:06:51.003522 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:51.003580 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 16:06:51.003856 18041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6274597ce8be486fba4a0613d099ea9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "939100169d094eeba3e04bb2ea5304ff"
I20260806 16:06:51.004062 18041 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6274597ce8be486fba4a0613d099ea9f in current term 2: Already voted for candidate 939100169d094eeba3e04bb2ea5304ff in this term.
I20260806 16:06:51.004328 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:51.004003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49546)
0806 16:06:51.004161 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:51.004219 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260806 16:06:51.004485 18005 leader_election.cc:334] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969): Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
I20260806 16:06:51.004568 18005 leader_election.cc:302] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 939100169d094eeba3e04bb2ea5304ff; no voters: 6274597ce8be486fba4a0613d099ea9f, 62f1112cc8e447e19ce70e0eab5a186b
I20260806 16:06:51.005048 18798 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 16:06:51.005167 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:51.004882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49556)
0806 16:06:51.005009 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:51.005070 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 16:06:51.005326 17734 leader_election.cc:334] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969): Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
I20260806 16:06:51.005554 17734 leader_election.cc:302] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6274597ce8be486fba4a0613d099ea9f; no voters: 62f1112cc8e447e19ce70e0eab5a186b, 939100169d094eeba3e04bb2ea5304ff
I20260806 16:06:51.005828 18797 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 16:06:51.463474 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:51.462750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.2:55871)
0806 16:06:51.463217 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:51.463307 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":234,"threads_started":1}
I20260806 16:06:51.467159 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:51.466646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.1:35097)
0806 16:06:51.466989 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:51.467061 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":131,"threads_started":1}
I20260806 16:06:51.474437 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:51.473882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.3:51585)
0806 16:06:51.474236 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:51.474313 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":153,"threads_started":1}
I20260806 16:06:52.465245 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:52.464532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.2:38449)
0806 16:06:52.465003 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:52.465093 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":112,"threads_started":1}
I20260806 16:06:52.468672 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:52.468049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.1:45045)
0806 16:06:52.468452 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:52.468549 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":116,"threads_started":1}
I20260806 16:06:52.476136 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:52.475306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.3:55819)
0806 16:06:52.475944 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:52.476013 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":170,"threads_started":1}
I20260806 16:06:53.036958 18822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:53.037115 18822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } }
I20260806 16:06:53.037654 18822 leader_election.cc:288] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969), 6274597ce8be486fba4a0613d099ea9f (127.8.49.61:38683)
I20260806 16:06:53.038319 17771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "939100169d094eeba3e04bb2ea5304ff" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6274597ce8be486fba4a0613d099ea9f" is_pre_election: true
I20260806 16:06:53.038506 17771 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939100169d094eeba3e04bb2ea5304ff in term 2.
I20260806 16:06:53.038816 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.037976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49834)
0806 16:06:53.038635 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.038725 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":95,"threads_started":1}
I20260806 16:06:53.038975 18006 leader_election.cc:302] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6274597ce8be486fba4a0613d099ea9f, 939100169d094eeba3e04bb2ea5304ff; no voters:
I20260806 16:06:53.039244 18822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 16:06:53.039301 18822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:53.039332 18822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 2 FOLLOWER]: Advancing to term 3
I20260806 16:06:53.040386 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.040051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49838)
0806 16:06:53.040219 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.040288 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 16:06:53.040558 18005 leader_election.cc:334] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969): Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
I20260806 16:06:53.040628 18822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } }
I20260806 16:06:53.041117 18822 leader_election.cc:288] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 3 election: Requested vote from peers 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969), 6274597ce8be486fba4a0613d099ea9f (127.8.49.61:38683)
I20260806 16:06:53.041816 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.041499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49848)
0806 16:06:53.041664 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.041744 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 16:06:53.041752 17771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "939100169d094eeba3e04bb2ea5304ff" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6274597ce8be486fba4a0613d099ea9f"
I20260806 16:06:53.041971 17771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 2 FOLLOWER]: Advancing to term 3
I20260806 16:06:53.042740 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.042501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49864)
0806 16:06:53.042628 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.042687 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 16:06:53.042917 18005 leader_election.cc:334] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969): Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
I20260806 16:06:53.043715 17771 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939100169d094eeba3e04bb2ea5304ff in term 3.
I20260806 16:06:53.044157 18006 leader_election.cc:302] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6274597ce8be486fba4a0613d099ea9f, 939100169d094eeba3e04bb2ea5304ff; no voters: 62f1112cc8e447e19ce70e0eab5a186b
I20260806 16:06:53.044371 18822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 3 FOLLOWER]: Leader election won for term 3
I20260806 16:06:53.044684 18822 raft_consensus.cc:686] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [term 3 LEADER]: Becoming Leader. State: Replica: 939100169d094eeba3e04bb2ea5304ff, State: Running, Role: LEADER
I20260806 16:06:53.045020 18822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } }
I20260806 16:06:53.046545 18826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 939100169d094eeba3e04bb2ea5304ff. Latest consensus state: current_term: 3 leader_uuid: "939100169d094eeba3e04bb2ea5304ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:53.046722 18826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:53.047119 18830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:53.050125 18830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:53.050592 18830 catalog_manager.cc:1259] Loaded cluster ID: 0b8b98af43d743aa824a44346403a0cc
I20260806 16:06:53.050658 18830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:53.051366 18830 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:53.051812 18830 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: Loaded TSK: 0
I20260806 16:06:53.052726 18830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:06:53.125344 18032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:53.129145 18032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:53.134196 17771 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [term 3 FOLLOWER]: Refusing update from remote peer 939100169d094eeba3e04bb2ea5304ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 16:06:53.134378 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.133851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49880)
0806 16:06:53.134028 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.134203 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 16:06:53.134969 18822 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 16:06:53.135515 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.135204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.0.0.1:49894)
0806 16:06:53.135376 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.135424 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260806 16:06:53.135691 18005 consensus_peers.cc:487] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff -> Peer 62f1112cc8e447e19ce70e0eab5a186b (127.8.49.62:45969): Couldn't send request to peer 62f1112cc8e447e19ce70e0eab5a186b. Status: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 16:06:53.137189 18833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 939100169d094eeba3e04bb2ea5304ff. Latest consensus state: current_term: 3 leader_uuid: "939100169d094eeba3e04bb2ea5304ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:53.137419 18833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:06:53.138746 18832 mvcc.cc:205] Tried to move back new op lower bound from 7315588764188659712 to 7315588763835473920. Current Snapshot: MvccSnapshot[applied={T|T < 7315588764188659712}]
I20260806 16:06:53.139031 18826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "939100169d094eeba3e04bb2ea5304ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:53.139210 18826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:53.140178 18833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "939100169d094eeba3e04bb2ea5304ff" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62f1112cc8e447e19ce70e0eab5a186b" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45969 } } peers { permanent_uuid: "6274597ce8be486fba4a0613d099ea9f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 38683 } attrs { promote: false } } peers { permanent_uuid: "939100169d094eeba3e04bb2ea5304ff" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 39957 } attrs { promote: false } } }
I20260806 16:06:53.140391 18833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6274597ce8be486fba4a0613d099ea9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:06:53.140859 18835 mvcc.cc:205] Tried to move back new op lower bound from 7315588764188659712 to 7315588763835473920. Current Snapshot: MvccSnapshot[applied={T|T < 7315588764188659712}]
I20260806 16:06:53.154317 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.149989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46658 (local address 127.8.49.2:44165)
0806 16:06:53.150560 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:53.150565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:53.150584 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:06:53.150668 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:53.150671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:53.150717 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:53.150829 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:53.151150 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.151838 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.153272 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.153718 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.153843 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:53.153917 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:53.154115 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 16:06:53.154196 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":94,"threads_started":1}
I20260806 16:06:53.154316 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.149154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:44165 (local address 127.0.0.1:46658)
0806 16:06:53.149545 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.149566 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:53.149696 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:53.150864 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:53.150867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:53.150886 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:53.151010 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:53.151019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.151976 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.151979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:53.153139 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:53.153149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.153847 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.153851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:53.153959 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:53.153988 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 16:06:53.154074 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 16:06:53.154162 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":30}
I20260806 16:06:53.155164 18841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.149448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:43059 (local address 127.0.0.1:59352)
0806 16:06:53.150166 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.150186 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:53.150301 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:53.151037 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:53.151041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:53.151062 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:53.151155 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:53.151160 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.152259 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.152263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:53.153598 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:53.153610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.154567 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.154571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:53.154680 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:53.154714 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 16:06:53.154768 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 16:06:53.154975 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":36,"thread_start_us":160,"threads_started":1}
I20260806 16:06:53.155357 18395 tablet_service.cc:1467] Processing CreateTablet for tablet 011586bcde9341799c7174d61b6f6864 (DEFAULT_TABLE table=test-table [id=7b8bc2c785264899802bd489e78e4a4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:53.155479 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.149785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40541 (local address 127.0.0.1:57222)
0806 16:06:53.151074 (+ 1289us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.151091 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:53.151286 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:53.151673 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:53.151677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:53.151698 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:53.151802 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:53.151807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.152839 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.152843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:53.153995 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:53.154004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.155044 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.155048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:53.155140 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:53.155178 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 16:06:53.155267 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 16:06:53.155356 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1215,"thread_start_us":100,"threads_started":1}
I20260806 16:06:53.155743 18395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 011586bcde9341799c7174d61b6f6864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:53.155776 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.150115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59352 (local address 127.8.49.3:43059)
0806 16:06:53.150548 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:53.150567 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:53.150589 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:06:53.150732 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:53.150736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:53.150808 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:53.151019 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:53.151289 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.152104 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.153771 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.154424 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.155404 (+ 980us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:53.155483 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:53.155528 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 16:06:53.155633 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":112,"threads_started":1}
I20260806 16:06:53.155893 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.149938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57222 (local address 127.8.49.1:40541)
0806 16:06:53.150560 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:53.150565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:53.151388 (+ 823us) server_negotiation.cc:407] Connection header received
0806 16:06:53.151462 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:53.151465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:53.151521 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:53.151622 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:53.151939 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.152690 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.154131 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.154769 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.154909 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:53.154984 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:53.155346 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 16:06:53.155499 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":115,"threads_started":1}
I20260806 16:06:53.156598 18197 tablet_service.cc:1467] Processing CreateTablet for tablet 011586bcde9341799c7174d61b6f6864 (DEFAULT_TABLE table=test-table [id=7b8bc2c785264899802bd489e78e4a4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:53.156921 18197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 011586bcde9341799c7174d61b6f6864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:53.158324 18848 tablet_bootstrap.cc:492] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501: Bootstrap starting.
I20260806 16:06:53.159500 18848 tablet_bootstrap.cc:654] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:53.160058 18850 tablet_bootstrap.cc:492] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2: Bootstrap starting.
I20260806 16:06:53.160719 18596 tablet_service.cc:1467] Processing CreateTablet for tablet 011586bcde9341799c7174d61b6f6864 (DEFAULT_TABLE table=test-table [id=7b8bc2c785264899802bd489e78e4a4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:53.161070 18596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 011586bcde9341799c7174d61b6f6864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:53.161245 18848 tablet_bootstrap.cc:492] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501: No bootstrap required, opened a new log
I20260806 16:06:53.161301 18850 tablet_bootstrap.cc:654] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:53.161367 18848 ts_tablet_manager.cc:1397] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:06:53.161864 18848 raft_consensus.cc:348] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } }
I20260806 16:06:53.162036 18848 raft_consensus.cc:374] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:53.162081 18848 raft_consensus.cc:729] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 420507bd250744c2b3a2a83baf2cb501, State: Initialized, Role: FOLLOWER
I20260806 16:06:53.162220 18848 consensus_queue.cc:260] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } }
I20260806 16:06:53.162621 18848 ts_tablet_manager.cc:1428] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:06:53.163975 18850 tablet_bootstrap.cc:492] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2: No bootstrap required, opened a new log
I20260806 16:06:53.164081 18850 ts_tablet_manager.cc:1397] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:53.164675 18850 raft_consensus.cc:348] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } }
I20260806 16:06:53.164821 18850 raft_consensus.cc:374] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:53.164857 18850 raft_consensus.cc:729] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 702e742c23214b40aa48256127f733f2, State: Initialized, Role: FOLLOWER
I20260806 16:06:53.165036 18850 consensus_queue.cc:260] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } }
I20260806 16:06:53.165438 18850 ts_tablet_manager.cc:1428] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:06:53.166227 18493 heartbeater.cc:503] Master 127.8.49.60:39957 was elected leader, sending a full tablet report...
I20260806 16:06:53.166594 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.165787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.1:34335)
0806 16:06:53.166381 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.166461 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":186,"threads_started":1}
I20260806 16:06:53.167857 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.163130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.2:49817)
0806 16:06:53.167653 (+ 4523us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.167730 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4436,"thread_start_us":142,"threads_started":1}
I20260806 16:06:53.168474 18299 heartbeater.cc:503] Master 127.8.49.60:39957 was elected leader, sending a full tablet report...
I20260806 16:06:53.168780 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.168477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.1:56547)
0806 16:06:53.168625 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.168698 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 16:06:53.168895 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.168594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.2:59207)
0806 16:06:53.168765 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.168817 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260806 16:06:53.169250 18858 tablet_bootstrap.cc:492] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17: Bootstrap starting.
I20260806 16:06:53.170583 18858 tablet_bootstrap.cc:654] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:53.172606 18858 tablet_bootstrap.cc:492] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17: No bootstrap required, opened a new log
I20260806 16:06:53.172729 18858 ts_tablet_manager.cc:1397] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:53.173302 18858 raft_consensus.cc:348] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } }
I20260806 16:06:53.173431 18858 raft_consensus.cc:374] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:53.173455 18858 raft_consensus.cc:729] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3dc20b5f304e69a6cf42f8baa83e17, State: Initialized, Role: FOLLOWER
I20260806 16:06:53.173596 18858 consensus_queue.cc:260] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } }
I20260806 16:06:53.173924 18858 ts_tablet_manager.cc:1428] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:06:53.175096 18868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.174429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.3:59085)
0806 16:06:53.174809 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.174967 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":176,"threads_started":1}
I20260806 16:06:53.175788 18868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.175562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.3:59907)
0806 16:06:53.175684 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.175737 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 16:06:53.176612 18694 heartbeater.cc:503] Master 127.8.49.60:39957 was elected leader, sending a full tablet report...
I20260806 16:06:53.323098 18855 raft_consensus.cc:482] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:53.323374 18855 raft_consensus.cc:504] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } }
I20260806 16:06:53.325350 18855 leader_election.cc:288] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c3dc20b5f304e69a6cf42f8baa83e17 (127.8.49.3:43059), 702e742c23214b40aa48256127f733f2 (127.8.49.1:40541)
I20260806 16:06:53.329975 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.325792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:42433 (local address 127.8.49.3:43059)
0806 16:06:53.325944 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:53.325951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:53.326094 (+ 143us) server_negotiation.cc:407] Connection header received
0806 16:06:53.326294 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:53.326298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:53.326368 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:53.326471 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:53.326860 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.327486 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.328753 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.329241 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.329382 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:53.329474 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:53.329711 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 16:06:53.329804 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 16:06:53.329975 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.325684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:43059 (local address 127.8.49.2:42433)
0806 16:06:53.325941 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.325960 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:53.326117 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:53.326493 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:53.326497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:53.326518 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:53.326693 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:53.326699 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.327630 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.327635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:53.328595 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:53.328605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.329394 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.329398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:53.329554 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:53.329579 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 16:06:53.329685 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 16:06:53.329791 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260806 16:06:53.330546 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.326189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:58637 (local address 127.8.49.1:40541)
0806 16:06:53.326451 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:53.326457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:53.326796 (+ 339us) server_negotiation.cc:407] Connection header received
0806 16:06:53.327061 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:53.327064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:53.327117 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:53.327210 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:53.327553 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.328140 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.329341 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.329786 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.329934 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:53.330010 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:53.330276 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 16:06:53.330381 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 16:06:53.330807 18624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011586bcde9341799c7174d61b6f6864" candidate_uuid: "420507bd250744c2b3a2a83baf2cb501" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" is_pre_election: true
I20260806 16:06:53.331067 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.326200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40541 (local address 127.8.49.2:58637)
0806 16:06:53.326671 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.326685 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:53.326896 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:53.327238 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:53.327241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:53.327259 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:53.327426 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:53.327432 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.328280 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.328285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:53.329191 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:53.329202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.329945 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.329949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:53.330086 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:53.330117 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 16:06:53.330756 (+ 639us) client_negotiation.cc:240] Negotiation successful
0806 16:06:53.330926 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":164,"threads_started":1}
I20260806 16:06:53.331259 18624 raft_consensus.cc:2454] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 420507bd250744c2b3a2a83baf2cb501 in term 0.
I20260806 16:06:53.331724 18231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011586bcde9341799c7174d61b6f6864" candidate_uuid: "420507bd250744c2b3a2a83baf2cb501" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "702e742c23214b40aa48256127f733f2" is_pre_election: true
I20260806 16:06:53.331899 18231 raft_consensus.cc:2454] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 420507bd250744c2b3a2a83baf2cb501 in term 0.
I20260806 16:06:53.331861 18330 leader_election.cc:302] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3dc20b5f304e69a6cf42f8baa83e17, 420507bd250744c2b3a2a83baf2cb501; no voters:
I20260806 16:06:53.332247 18855 raft_consensus.cc:2781] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:53.332329 18855 raft_consensus.cc:482] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:53.332365 18855 raft_consensus.cc:3037] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:53.333793 18855 raft_consensus.cc:504] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } }
I20260806 16:06:53.334298 18855 leader_election.cc:288] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [CANDIDATE]: Term 1 election: Requested vote from peers 2c3dc20b5f304e69a6cf42f8baa83e17 (127.8.49.3:43059), 702e742c23214b40aa48256127f733f2 (127.8.49.1:40541)
I20260806 16:06:53.334853 18624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011586bcde9341799c7174d61b6f6864" candidate_uuid: "420507bd250744c2b3a2a83baf2cb501" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17"
I20260806 16:06:53.334909 18231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011586bcde9341799c7174d61b6f6864" candidate_uuid: "420507bd250744c2b3a2a83baf2cb501" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "702e742c23214b40aa48256127f733f2"
I20260806 16:06:53.335001 18624 raft_consensus.cc:3037] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:53.335089 18231 raft_consensus.cc:3037] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:53.336306 18624 raft_consensus.cc:2454] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 420507bd250744c2b3a2a83baf2cb501 in term 1.
I20260806 16:06:53.336324 18231 raft_consensus.cc:2454] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 420507bd250744c2b3a2a83baf2cb501 in term 1.
I20260806 16:06:53.336858 18332 leader_election.cc:302] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 420507bd250744c2b3a2a83baf2cb501, 702e742c23214b40aa48256127f733f2; no voters:
I20260806 16:06:53.337141 18855 raft_consensus.cc:2781] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:53.337486 18855 raft_consensus.cc:686] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 1 LEADER]: Becoming Leader. State: Replica: 420507bd250744c2b3a2a83baf2cb501, State: Running, Role: LEADER
I20260806 16:06:53.337865 18855 consensus_queue.cc:237] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } } peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } }
I20260806 16:06:53.338640 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.338202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.2:60925)
0806 16:06:53.338470 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.338551 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 16:06:53.341306 18033 catalog_manager.cc:5495] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 reported cstate change: term changed from 0 to 1, leader changed from <none> to 420507bd250744c2b3a2a83baf2cb501 (127.8.49.2). New cstate: current_term: 1 leader_uuid: "420507bd250744c2b3a2a83baf2cb501" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "420507bd250744c2b3a2a83baf2cb501" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 44165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 } health_report { overall_health: UNKNOWN } } }
I20260806 16:06:53.362123 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.357355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.8.49.2:44165)
0806 16:06:53.357558 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:53.357562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:53.357715 (+ 153us) server_negotiation.cc:407] Connection header received
0806 16:06:53.357875 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:53.357878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:53.357943 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:53.358062 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:53.358477 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.359073 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.360307 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:53.360572 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:53.360665 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:53.361067 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:53.361673 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:53.361780 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:53.361940 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 16:06:53.362003 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260806 16:06:53.366387 18231 raft_consensus.cc:1261] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2 [term 1 FOLLOWER]: Refusing update from remote peer 420507bd250744c2b3a2a83baf2cb501: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:53.366677 18624 raft_consensus.cc:1261] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 1 FOLLOWER]: Refusing update from remote peer 420507bd250744c2b3a2a83baf2cb501: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:53.367198 18855 consensus_queue.cc:1035] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "702e742c23214b40aa48256127f733f2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:53.367475 18871 consensus_queue.cc:1035] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3dc20b5f304e69a6cf42f8baa83e17" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 43059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:53.369889 18868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.369574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.3:50551)
0806 16:06:53.369698 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.369771 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 16:06:53.376438 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.370895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.1:47075)
0806 16:06:53.376075 (+ 5180us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.376171 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5086}
I20260806 16:06:53.380100 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.379698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.2:57855)
0806 16:06:53.379904 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.379993 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 16:06:53.384480 18868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.384146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.3:55389)
0806 16:06:53.384321 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.384385 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260806 16:06:53.385833 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.385146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.2:50245)
0806 16:06:53.385683 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.385746 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444}
I20260806 16:06:53.388919 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.387280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45969 (local address 127.8.49.1:59841)
0806 16:06:53.388674 (+ 1394us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.388761 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1286}
I20260806 16:06:53.395241 18033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:53.395529 18033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:53.402761 18033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 939100169d094eeba3e04bb2ea5304ff: Sending DeleteTablet for 3 replicas of tablet 011586bcde9341799c7174d61b6f6864
I20260806 16:06:53.403970 18596 tablet_service.cc:1514] Processing DeleteTablet for tablet 011586bcde9341799c7174d61b6f6864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:53 UTC) from {username='oryx', principal='oryx/127.8.49.60@KRBTEST.COM'} at 127.0.0.1:59352
I20260806 16:06:53.404595 18395 tablet_service.cc:1514] Processing DeleteTablet for tablet 011586bcde9341799c7174d61b6f6864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:53 UTC) from {username='oryx', principal='oryx/127.8.49.60@KRBTEST.COM'} at 127.0.0.1:46658
I20260806 16:06:53.404886 18197 tablet_service.cc:1514] Processing DeleteTablet for tablet 011586bcde9341799c7174d61b6f6864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:53 UTC) from {username='oryx', principal='oryx/127.8.49.60@KRBTEST.COM'} at 127.0.0.1:57222
I20260806 16:06:53.404963 18888 tablet_replica.cc:330] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17: stopping tablet replica
I20260806 16:06:53.405171 18888 raft_consensus.cc:2229] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:53.405378 18888 raft_consensus.cc:2258] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:53.405401 18890 tablet_replica.cc:330] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2: stopping tablet replica
I20260806 16:06:53.405586 18890 raft_consensus.cc:2229] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:53.405771 18890 raft_consensus.cc:2258] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:53.406426 18888 ts_tablet_manager.cc:1905] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:53.406579 18890 ts_tablet_manager.cc:1905] T 011586bcde9341799c7174d61b6f6864 P 702e742c23214b40aa48256127f733f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:53.407701 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 17967
I20260806 16:06:53.415719 18888 ts_tablet_manager.cc:1918] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:53.415867 18888 log.cc:1196] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/011586bcde9341799c7174d61b6f6864
I20260806 16:06:53.416261 18888 ts_tablet_manager.cc:1939] T 011586bcde9341799c7174d61b6f6864 P 2c3dc20b5f304e69a6cf42f8baa83e17: Deleting consensus metadata
I20260806 16:06:53.419060 18002 catalog_manager.cc:4841] TS 2c3dc20b5f304e69a6cf42f8baa83e17 (127.8.49.3:43059): tablet 011586bcde9341799c7174d61b6f6864 (table test-table [id=7b8bc2c785264899802bd489e78e4a4f]) successfully deleted
I20260806 16:06:53.419754 18889 tablet_replica.cc:330] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501: stopping tablet replica
I20260806 16:06:53.420008 18889 raft_consensus.cc:2229] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:53.420882 18889 raft_consensus.cc:2258] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:53.422008 18889 ts_tablet_manager.cc:1905] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:53.425282 18889 ts_tablet_manager.cc:1918] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:53.425405 18889 log.cc:1196] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/011586bcde9341799c7174d61b6f6864
I20260806 16:06:53.425690 18889 ts_tablet_manager.cc:1939] T 011586bcde9341799c7174d61b6f6864 P 420507bd250744c2b3a2a83baf2cb501: Deleting consensus metadata
I20260806 16:06:53.426926 18005 catalog_manager.cc:4841] TS 420507bd250744c2b3a2a83baf2cb501 (127.8.49.2:44165): tablet 011586bcde9341799c7174d61b6f6864 (table test-table [id=7b8bc2c785264899802bd489e78e4a4f]) successfully deleted
I20260806 16:06:53.428138 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 18309
I20260806 16:06:53.435983 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.428446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40541 (local address 127.0.0.1:57228)
0806 16:06:53.435697 (+ 7251us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.435816 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7043}
W20260806 16:06:53.436563 18005 catalog_manager.cc:4584] TS 702e742c23214b40aa48256127f733f2 (127.8.49.1:40541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 011586bcde9341799c7174d61b6f6864: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40541: connect: Connection refused (error 111)
I20260806 16:06:53.445977 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 18502
I20260806 16:06:53.475103 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 17545
I20260806 16:06:53.489372 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.488795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40541 (local address 127.0.0.1:57232)
0806 16:06:53.489089 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.489221 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260806 16:06:53.490552 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:53.490222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40541 (local address 127.0.0.1:57244)
0806 16:06:53.490380 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:53.490463 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 16:06:53.493718 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 17701
2026-08-06T16:06:53Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9055 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:53 dist-test-slave-7z32 krb5kdc[18893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:53 dist-test-slave-7z32 krb5kdc[18893](info): set up 2 sockets
Aug 06 16:06:53 dist-test-slave-7z32 krb5kdc[18893](info): commencing operation
krb5kdc: starting...
W20260806 16:06:54.850970 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.002s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:54 dist-test-slave-7z32 krb5kdc[18893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.62.keytab.
Entry for principal oryx/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.62.keytab.
I20260806 16:06:55.030625 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42755
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.62.keytab
--principal=oryx/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:42755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:55.164731 18913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:55.165135 18913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:55.165187 18913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:55.169837 18913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:55.169937 18913 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 16:06:55.169957 18913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:55.169976 18913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:55.169994 18913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:55.170011 18913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:55.176399 18913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:42755
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.62.keytab
--principal=oryx/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:55.178406 18913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:55.180150 18913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:06:55.193318 18913 server_base.cc:1034] running on GCE node
W20260806 16:06:55.193272 18928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:55.193404 18925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:55.193341 18926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:55.194411 18913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:55.196309 18913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:55.197575 18913 hybrid_clock.cc:648] HybridClock initialized: now 1786032415197533 us; error 62 us; skew 500 ppm
Aug 06 16:06:55 dist-test-slave-7z32 krb5kdc[18893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032415, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:55.201581 18913 init.cc:377] Logged in from keytab as oryx/127.8.49.62@KRBTEST.COM (short username oryx)
I20260806 16:06:55.203014 18913 webserver.cc:460] Webserver started at http://127.8.49.62:38891/ using document root <none> and password file <none>
I20260806 16:06:55.203917 18913 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:55.204046 18913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:55.204641 18913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:55.207849 18913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "849154fda66c4e3ba2e32a77a50706e6"
format_stamp: "Formatted at 2026-08-06 16:06:55 on dist-test-slave-7z32"
server_key: "fb00ff8aee4fc8bcee61d05f44aa7e5f"
server_key_iv: "8c0b464cc801ee24fccecb85415d55b3"
server_key_version: "encryptionkey@0"
I20260806 16:06:55.208652 18913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "849154fda66c4e3ba2e32a77a50706e6"
format_stamp: "Formatted at 2026-08-06 16:06:55 on dist-test-slave-7z32"
server_key: "fb00ff8aee4fc8bcee61d05f44aa7e5f"
server_key_iv: "8c0b464cc801ee24fccecb85415d55b3"
server_key_version: "encryptionkey@0"
I20260806 16:06:55.214576 18913 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 16:06:55.218317 18939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:55.220479 18913 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:55.220701 18913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "849154fda66c4e3ba2e32a77a50706e6"
format_stamp: "Formatted at 2026-08-06 16:06:55 on dist-test-slave-7z32"
server_key: "fb00ff8aee4fc8bcee61d05f44aa7e5f"
server_key_iv: "8c0b464cc801ee24fccecb85415d55b3"
server_key_version: "encryptionkey@0"
I20260806 16:06:55.220880 18913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:55.240556 18913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:55.245548 18913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:55.245898 18913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:55.258400 18913 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:42755
I20260806 16:06:55.258468 19031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:42755 every 8 connection(s)
I20260806 16:06:55.260274 18913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:06:55.261106 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 18913
I20260806 16:06:55.261273 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:06:55.261620 8388 external_mini_cluster.cc:1428] Setting key d12ad5a0c465e296c44bfa756e805475
I20260806 16:06:55.265095 19032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:55.273190 19032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6: Bootstrap starting.
Aug 06 16:06:55 dist-test-slave-7z32 krb5kdc[18893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
I20260806 16:06:55.276428 19032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:55.277402 19032 log.cc:824] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:55.280663 19032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6: No bootstrap required, opened a new log
I20260806 16:06:55.285130 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:55.264068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35626 (local address 127.8.49.62:42755)
0806 16:06:55.264772 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:55.264796 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:55.264872 (+ 76us) server_negotiation.cc:407] Connection header received
0806 16:06:55.265893 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:55.265921 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:55.266412 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:55.266924 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:55.268360 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.269962 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.271254 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.272007 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.272207 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:55.276393 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:55.276402 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:55.276455 (+ 53us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:55.276531 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:55.280497 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:55.281445 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:55.281452 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:55.281459 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:55.281639 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:55.282084 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:55.282088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:55.282090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:55.282703 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:55.282979 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:55.283583 (+ 604us) server_negotiation.cc:299] Negotiation successful
0806 16:06:55.283984 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":206,"threads_started":1}
I20260806 16:06:55.285061 19032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849154fda66c4e3ba2e32a77a50706e6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42755 } }
I20260806 16:06:55.285385 19032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:55.285537 19032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 849154fda66c4e3ba2e32a77a50706e6, State: Initialized, Role: FOLLOWER
I20260806 16:06:55.286047 19032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849154fda66c4e3ba2e32a77a50706e6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42755 } }
I20260806 16:06:55.286237 19032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:06:55.286355 19032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:06:55.286492 19032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:55.288033 19032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849154fda66c4e3ba2e32a77a50706e6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42755 } }
I20260806 16:06:55.288683 19032 leader_election.cc:302] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 849154fda66c4e3ba2e32a77a50706e6; no voters:
I20260806 16:06:55.289292 19032 leader_election.cc:288] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:06:55.289812 19042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:55.290174 19042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [term 1 LEADER]: Becoming Leader. State: Replica: 849154fda66c4e3ba2e32a77a50706e6, State: Running, Role: LEADER
I20260806 16:06:55.290607 19042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849154fda66c4e3ba2e32a77a50706e6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42755 } }
I20260806 16:06:55.291122 19032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:06:55.293020 19044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 849154fda66c4e3ba2e32a77a50706e6. Latest consensus state: current_term: 1 leader_uuid: "849154fda66c4e3ba2e32a77a50706e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849154fda66c4e3ba2e32a77a50706e6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42755 } } }
I20260806 16:06:55.293228 19044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:55.293581 19043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "849154fda66c4e3ba2e32a77a50706e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849154fda66c4e3ba2e32a77a50706e6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42755 } } }
I20260806 16:06:55.293733 19043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6 [sys.catalog]: This master's current role is: LEADER
I20260806 16:06:55.294144 19055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:06:55.297969 19055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:06:55.307860 19055 catalog_manager.cc:1347] Generated new cluster ID: 6407305d774a4ea6bc786ace3e2bce54
I20260806 16:06:55.308001 19055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:06:55.324652 19055 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:06:55.326275 19055 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:06:55.331907 19055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6: Generated new TSK 0
I20260806 16:06:55.332931 19055 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab.
Entry for principal oryx/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab.
I20260806 16:06:55.387723 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:42755
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab
--principal=oryx/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:55.539486 19068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:55.539909 19068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:55.539966 19068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:55.545188 19068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:55.545322 19068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:55.545375 19068 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:06:55.545416 19068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:06:55.553577 19068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab
--principal=oryx/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.49.62:42755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:55.555356 19068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:55.556944 19068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:06:55.571550 19068 server_base.cc:1034] running on GCE node
W20260806 16:06:55.571380 19077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:55.571380 19076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:55.571453 19080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:55.572503 19068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:55.573606 19068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:55.574934 19068 hybrid_clock.cc:648] HybridClock initialized: now 1786032415574906 us; error 48 us; skew 500 ppm
Aug 06 16:06:55 dist-test-slave-7z32 krb5kdc[18893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032415, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:55.579428 19068 init.cc:377] Logged in from keytab as oryx/127.8.49.1@KRBTEST.COM (short username oryx)
I20260806 16:06:55.581041 19068 webserver.cc:460] Webserver started at http://127.8.49.1:36743/ using document root <none> and password file <none>
I20260806 16:06:55.582118 19068 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:55.582268 19068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:55.582903 19068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:55.586586 19068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "eb5f2f647cde43dc908cacc2885682a9"
format_stamp: "Formatted at 2026-08-06 16:06:55 on dist-test-slave-7z32"
server_key: "6b2d61c5813d6bb6587475ae43c7314c"
server_key_iv: "f50764cdd59dfe30789f97dfc565d8a4"
server_key_version: "encryptionkey@0"
I20260806 16:06:55.587491 19068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "eb5f2f647cde43dc908cacc2885682a9"
format_stamp: "Formatted at 2026-08-06 16:06:55 on dist-test-slave-7z32"
server_key: "6b2d61c5813d6bb6587475ae43c7314c"
server_key_iv: "f50764cdd59dfe30789f97dfc565d8a4"
server_key_version: "encryptionkey@0"
I20260806 16:06:55.593966 19068 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:06:55.597894 19091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:55.599828 19068 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:55.599987 19068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "eb5f2f647cde43dc908cacc2885682a9"
format_stamp: "Formatted at 2026-08-06 16:06:55 on dist-test-slave-7z32"
server_key: "6b2d61c5813d6bb6587475ae43c7314c"
server_key_iv: "f50764cdd59dfe30789f97dfc565d8a4"
server_key_version: "encryptionkey@0"
I20260806 16:06:55.600193 19068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:55.625981 19068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:55.630419 19068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:55.630610 19068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:55.632447 19068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:55.632517 19068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:55.632575 19068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:55.632642 19068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:55.645725 19068 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:41253
I20260806 16:06:55.645855 19259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:41253 every 8 connection(s)
Aug 06 16:06:55 dist-test-slave-7z32 krb5kdc[18893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032415, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
I20260806 16:06:55.647113 19068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:06:55.651517 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 19068
I20260806 16:06:55.651738 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:06:55.652145 8388 external_mini_cluster.cc:1428] Setting key 41074befab17419c725e5f8469ed1b66
I20260806 16:06:55.652252 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:55.634460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:39661 (local address 127.8.49.62:42755)
0806 16:06:55.634646 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:55.634698 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:55.635996 (+ 1298us) server_negotiation.cc:407] Connection header received
0806 16:06:55.638032 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:55.638039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:55.638136 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:55.638284 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:55.640221 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.641192 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.643159 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.643600 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.643775 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:55.647946 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:55.647951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:55.647954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:55.647992 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:55.649674 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:55.650373 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:55.650376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:55.650378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:55.650544 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:55.650969 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:55.650972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:55.650974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:55.651200 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:55.651316 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:55.651892 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 16:06:55.652058 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:06:55.653254 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:55.635020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42755 (local address 127.8.49.1:39661)
0806 16:06:55.635732 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:55.635827 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:55.637655 (+ 1828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:55.638601 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:55.638616 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:55.639235 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:55.639962 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:55.639991 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.641420 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.641425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:55.642994 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:55.643010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.643956 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.643964 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:55.644260 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:55.645126 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:55.645149 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:55.647672 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:55.649869 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:55.649876 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:55.649894 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:55.650243 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:55.650653 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:55.650657 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:55.650659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:55.650863 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:55.651317 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:55.651327 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:55.651621 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 16:06:55.651832 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 16:06:55.652215 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":160,"threads_started":1}
I20260806 16:06:55.654865 19260 heartbeater.cc:347] Connected to a master server at 127.8.49.62:42755
I20260806 16:06:55.655151 19260 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:55.656454 19260 heartbeater.cc:511] Master 127.8.49.62:42755 requested a full tablet report, sending...
I20260806 16:06:55.658980 18960 ts_manager.cc:194] Registered new tserver with Master: eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1:41253)
I20260806 16:06:55.660879 18960 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:39661
I20260806 16:06:55.669598 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:55.658018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:50039 (local address 127.8.49.62:42755)
0806 16:06:55.658315 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:55.658322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:55.658762 (+ 440us) server_negotiation.cc:407] Connection header received
0806 16:06:55.659061 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:55.659066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:55.659141 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:55.659204 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:55.659923 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.660945 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.662519 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.662984 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.663166 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:55.665304 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:55.665309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:55.665311 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:55.665343 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:55.667279 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:55.668004 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:55.668007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:55.668008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:55.668108 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:55.668538 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:55.668541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:55.668543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:55.668768 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:55.668873 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:55.669326 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 16:06:55.669457 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":182,"server-negotiator.queue_time_us":56}
WARNING: no policy specified for oryx/127.8.49.2@KRBTEST.COM; defaulting to no policy
I20260806 16:06:55.670117 19270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:55.657929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42755 (local address 127.8.49.1:50039)
0806 16:06:55.658613 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:55.658633 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:55.658914 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:55.659393 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:55.659395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:55.659643 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:55.659780 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:55.659786 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.661168 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.661176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:55.662329 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:55.662348 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.663198 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.663206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:55.663366 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:55.664358 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:55.664390 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:55.665072 (+ 682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:55.667462 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:55.667469 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:55.667474 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:55.667851 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:55.668254 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:55.668259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:55.668263 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:55.668385 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:55.668911 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:55.668917 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:55.669086 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 16:06:55.669230 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 16:06:55.669544 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":151,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab.
Entry for principal oryx/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab.
I20260806 16:06:55.701805 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:42755
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab
--principal=oryx/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:55.871335 19274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:55.871749 19274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:55.871804 19274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:55.877014 19274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:55.877156 19274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:55.877214 19274 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:06:55.877259 19274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:06:55.884859 19274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab
--principal=oryx/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.49.62:42755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:55.886472 19274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:55.887854 19274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:55.904385 19284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:55.904402 19283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:55.904495 19274 server_base.cc:1034] running on GCE node
W20260806 16:06:55.904412 19287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:55.905555 19274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:55.906766 19274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:55.908177 19274 hybrid_clock.cc:648] HybridClock initialized: now 1786032415908128 us; error 82 us; skew 500 ppm
Aug 06 16:06:55 dist-test-slave-7z32 krb5kdc[18893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032415, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:55.912819 19274 init.cc:377] Logged in from keytab as oryx/127.8.49.2@KRBTEST.COM (short username oryx)
I20260806 16:06:55.914397 19274 webserver.cc:460] Webserver started at http://127.8.49.2:43551/ using document root <none> and password file <none>
I20260806 16:06:55.915382 19274 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:55.915509 19274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:55.915836 19274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:55.919174 19274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "455f7250bda648c590c6c999665d81d5"
format_stamp: "Formatted at 2026-08-06 16:06:55 on dist-test-slave-7z32"
server_key: "4d0b9129a670723763bba35240b2e0bf"
server_key_iv: "f4a2e9dd371c72bfee7ba8aa959f79ad"
server_key_version: "encryptionkey@0"
I20260806 16:06:55.920011 19274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "455f7250bda648c590c6c999665d81d5"
format_stamp: "Formatted at 2026-08-06 16:06:55 on dist-test-slave-7z32"
server_key: "4d0b9129a670723763bba35240b2e0bf"
server_key_iv: "f4a2e9dd371c72bfee7ba8aa959f79ad"
server_key_version: "encryptionkey@0"
I20260806 16:06:55.926668 19274 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:06:55.930986 19296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:55.933051 19274 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:55.933316 19274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "455f7250bda648c590c6c999665d81d5"
format_stamp: "Formatted at 2026-08-06 16:06:55 on dist-test-slave-7z32"
server_key: "4d0b9129a670723763bba35240b2e0bf"
server_key_iv: "f4a2e9dd371c72bfee7ba8aa959f79ad"
server_key_version: "encryptionkey@0"
I20260806 16:06:55.933506 19274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:55.947222 19274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:55.952224 19274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:55.952512 19274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:55.954660 19274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:55.954748 19274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:55.954896 19274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:55.954941 19274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 06 16:06:55 dist-test-slave-7z32 krb5kdc[18893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032415, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
I20260806 16:06:55.970748 19274 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:37511
I20260806 16:06:55.970894 19464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:37511 every 8 connection(s)
I20260806 16:06:55.972287 19274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:06:55.974856 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:55.956073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:57207 (local address 127.8.49.62:42755)
0806 16:06:55.956223 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:55.956227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:55.957804 (+ 1577us) server_negotiation.cc:407] Connection header received
0806 16:06:55.959203 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:55.959206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:55.959269 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:55.959379 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:55.961956 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.962710 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.964330 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.964656 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.964768 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:55.969266 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:55.969271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:55.969273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:55.969312 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:55.971467 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:55.972377 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:55.972381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:55.972383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:55.972514 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:55.973142 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:55.973147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:55.973149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:55.973459 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:55.973641 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:55.974430 (+ 789us) server_negotiation.cc:299] Negotiation successful
0806 16:06:55.974597 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:06:55.975337 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 19274
I20260806 16:06:55.975509 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:06:55.975801 19308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:55.956839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42755 (local address 127.8.49.2:57207)
0806 16:06:55.957589 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:55.957659 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:55.958923 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:55.959835 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:55.959855 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:55.960572 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:55.961619 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:55.961654 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.962926 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.962931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:55.964170 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:55.964185 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.964965 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.964974 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:55.965358 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:55.966324 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:55.966355 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:55.969024 (+ 2669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:55.971806 (+ 2782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:55.971814 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:55.971837 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:55.972195 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:55.972675 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:55.972678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:55.972679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:55.972853 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:55.973655 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:55.973665 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:55.973972 (+ 307us) client_negotiation.cc:769] Sending connection context
0806 16:06:55.974363 (+ 391us) client_negotiation.cc:240] Negotiation successful
0806 16:06:55.974764 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":205,"threads_started":1}
I20260806 16:06:55.975934 8388 external_mini_cluster.cc:1428] Setting key 6721bb038c5a581d499189786a98ca95
I20260806 16:06:55.977959 19465 heartbeater.cc:347] Connected to a master server at 127.8.49.62:42755
I20260806 16:06:55.978276 19465 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:55.979269 19465 heartbeater.cc:511] Master 127.8.49.62:42755 requested a full tablet report, sending...
I20260806 16:06:55.980860 18959 ts_manager.cc:194] Registered new tserver with Master: 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511)
I20260806 16:06:55.981917 18959 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:57207
I20260806 16:06:55.993916 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:55.981426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:44565 (local address 127.8.49.62:42755)
0806 16:06:55.981651 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:55.981659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:55.983287 (+ 1628us) server_negotiation.cc:407] Connection header received
0806 16:06:55.983472 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:55.983475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:55.983534 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:55.983631 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:55.984291 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.984863 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.986608 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.986901 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.987025 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:55.988826 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:55.988829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:55.988831 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:55.988863 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:55.990699 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:55.991732 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:55.991736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:55.991747 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:55.991860 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:55.992399 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:55.992406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:55.992409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:55.992871 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:55.992980 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:55.993587 (+ 607us) server_negotiation.cc:299] Negotiation successful
0806 16:06:55.993759 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 16:06:55.994180 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:55.981292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42755 (local address 127.8.49.2:44565)
0806 16:06:55.983171 (+ 1879us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:55.983187 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:55.983352 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:55.983663 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:55.983666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:55.983931 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:55.984154 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:55.984162 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.985133 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.985139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:55.986236 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:55.986249 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:55.987041 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:55.987045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:55.987141 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:55.987844 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:55.987873 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:55.988553 (+ 680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:55.991036 (+ 2483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:55.991046 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:55.991053 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:55.991549 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:55.992002 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:55.992009 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:55.992011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:55.992154 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:55.993009 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:55.993015 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:55.993220 (+ 205us) client_negotiation.cc:769] Sending connection context
0806 16:06:55.993394 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 16:06:55.993815 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1744,"mutex_wait_us":57,"thread_start_us":178,"threads_started":1}
WARNING: no policy specified for oryx/127.8.49.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab.
Entry for principal oryx/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab.
I20260806 16:06:56.024462 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:42755
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab
--principal=oryx/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:56.198804 19475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:56.199288 19475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:56.199345 19475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:56.204188 19475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:56.204291 19475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:56.204341 19475 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 16:06:56.204432 19475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:06:56.211670 19475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab
--principal=oryx/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.49.62:42755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:56.213287 19475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:56.214700 19475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:56.230304 19481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:56.230312 19482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:56.230309 19486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:56.232682 19475 server_base.cc:1034] running on GCE node
I20260806 16:06:56.233390 19475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:56.234308 19475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:56.235662 19475 hybrid_clock.cc:648] HybridClock initialized: now 1786032416235621 us; error 73 us; skew 500 ppm
Aug 06 16:06:56 dist-test-slave-7z32 krb5kdc[18893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032416, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:06:56.239696 19475 init.cc:377] Logged in from keytab as oryx/127.8.49.3@KRBTEST.COM (short username oryx)
I20260806 16:06:56.241609 19475 webserver.cc:460] Webserver started at http://127.8.49.3:36133/ using document root <none> and password file <none>
I20260806 16:06:56.242560 19475 fs_manager.cc:359] Metadata directory not provided
I20260806 16:06:56.242678 19475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:06:56.243068 19475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:06:56.246301 19475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "83ed5b21cd654ed281efae740a991a75"
format_stamp: "Formatted at 2026-08-06 16:06:56 on dist-test-slave-7z32"
server_key: "418e30a789f89ff927462764783f8778"
server_key_iv: "4f0fe53b916bce4829d32cc4536e80f7"
server_key_version: "encryptionkey@0"
I20260806 16:06:56.247404 19475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "83ed5b21cd654ed281efae740a991a75"
format_stamp: "Formatted at 2026-08-06 16:06:56 on dist-test-slave-7z32"
server_key: "418e30a789f89ff927462764783f8778"
server_key_iv: "4f0fe53b916bce4829d32cc4536e80f7"
server_key_version: "encryptionkey@0"
I20260806 16:06:56.253856 19475 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:06:56.257941 19495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:56.260121 19475 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:56.260347 19475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "83ed5b21cd654ed281efae740a991a75"
format_stamp: "Formatted at 2026-08-06 16:06:56 on dist-test-slave-7z32"
server_key: "418e30a789f89ff927462764783f8778"
server_key_iv: "4f0fe53b916bce4829d32cc4536e80f7"
server_key_version: "encryptionkey@0"
I20260806 16:06:56.260527 19475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:06:56.283495 19475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:06:56.289443 19475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:06:56.289739 19475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:06:56.292249 19475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:06:56.292430 19475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:06:56.292519 19475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:06:56.292579 19475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 16:06:56 dist-test-slave-7z32 krb5kdc[18893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032416, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
I20260806 16:06:56.312047 19475 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:35131
I20260806 16:06:56.312162 19659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:35131 every 8 connection(s)
I20260806 16:06:56.313612 19475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:06:56.315251 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.293291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:40963 (local address 127.8.49.62:42755)
0806 16:06:56.293514 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:56.293522 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:56.294739 (+ 1217us) server_negotiation.cc:407] Connection header received
0806 16:06:56.296104 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:56.296113 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:56.296225 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:56.296458 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:56.298251 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.298962 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.300490 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.300733 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.300895 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.308084 (+ 7189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:56.308096 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:56.308103 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:56.308159 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:56.310893 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:56.312018 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:56.312023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:56.312025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:56.312180 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:56.312778 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:56.312783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:56.312785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:56.313255 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:56.313389 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:56.314817 (+ 1428us) server_negotiation.cc:299] Negotiation successful
0806 16:06:56.315063 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 16:06:56.317087 19514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.293854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42755 (local address 127.8.49.3:40963)
0806 16:06:56.294504 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:56.294563 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:56.295704 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:56.296709 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:56.296719 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:56.297379 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:56.298034 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.298052 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.299149 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.299155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.300326 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.300340 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.301041 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.301052 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.301462 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.302698 (+ 1236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:56.302722 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:56.307555 (+ 4833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:56.311215 (+ 3660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:56.311225 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:56.311247 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:56.311818 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:56.312344 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:56.312349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:56.312351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:56.312580 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:56.313431 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:56.313439 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:56.314069 (+ 630us) client_negotiation.cc:769] Sending connection context
0806 16:06:56.314915 (+ 846us) client_negotiation.cc:240] Negotiation successful
0806 16:06:56.315653 (+ 738us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":211,"threads_started":1}
I20260806 16:06:56.319312 19660 heartbeater.cc:347] Connected to a master server at 127.8.49.62:42755
I20260806 16:06:56.319388 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 19475
I20260806 16:06:56.319576 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:06:56.319763 19660 heartbeater.cc:464] Registering TS with master...
I20260806 16:06:56.320029 8388 external_mini_cluster.cc:1428] Setting key 6ba41a8da3d2b5d30d6c0d4e5215ad52
I20260806 16:06:56.320657 19660 heartbeater.cc:511] Master 127.8.49.62:42755 requested a full tablet report, sending...
I20260806 16:06:56.322531 18959 ts_manager.cc:194] Registered new tserver with Master: 83ed5b21cd654ed281efae740a991a75 (127.8.49.3:35131)
I20260806 16:06:56.323446 18959 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:40963
I20260806 16:06:56.323653 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:06:56.334872 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.323110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:41741 (local address 127.8.49.62:42755)
0806 16:06:56.323301 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:56.323309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:56.323791 (+ 482us) server_negotiation.cc:407] Connection header received
0806 16:06:56.324000 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:56.324003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:56.324057 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:56.324117 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:56.325105 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.325929 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.327742 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.328380 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.328591 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.330691 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:56.330696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:56.330698 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:56.330782 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:56.332748 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:56.333351 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:56.333360 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:56.333361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:56.333449 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:56.333780 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:56.333783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:56.333785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:56.334037 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:56.334158 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:56.334487 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 16:06:56.334636 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 16:06:56.334877 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.323228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42755 (local address 127.8.49.3:41741)
0806 16:06:56.323680 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:56.323702 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:56.323890 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:56.324353 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:56.324358 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:56.324698 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:56.324906 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.324919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.326169 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.326176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.327523 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.327541 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.328630 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.328667 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.328769 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.329782 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:56.329807 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:56.330441 (+ 634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:56.332938 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:56.332943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:56.332945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:56.333222 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:56.333555 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:56.333559 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:56.333561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:56.333659 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:56.334147 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:56.334151 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:56.334280 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 16:06:56.334411 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 16:06:56.334640 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":137,"threads_started":1}
Aug 06 16:06:56 dist-test-slave-7z32 krb5kdc[18893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032416, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 16:06:56 dist-test-slave-7z32 krb5kdc[18893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:06:56 dist-test-slave-7z32 krb5kdc[18893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:06:56.363472 19038 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:06:56.354150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35630 (local address 127.8.49.62:42755)
0806 16:06:56.354323 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:56.354326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:56.354344 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:06:56.354536 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:56.354539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:56.354596 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:56.354704 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:56.356139 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.356731 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.358014 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.358269 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.358389 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.363226 (+ 4837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35630: BlockingRecv error: recv got EOF from 127.0.0.1:35630 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 16:06:56 dist-test-slave-7z32 krb5kdc[18893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032416, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
I20260806 16:06:56.378715 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.365791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35636 (local address 127.8.49.62:42755)
0806 16:06:56.366019 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:56.366028 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:56.366273 (+ 245us) server_negotiation.cc:407] Connection header received
0806 16:06:56.366390 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:56.366393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:56.366461 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:56.366559 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:56.367828 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.368528 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.369575 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.369930 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.370049 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.373002 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:56.373007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:56.373009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:56.373049 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:56.375193 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:56.376033 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:56.376036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:56.376038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:56.376153 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:56.376844 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:56.376848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:56.376849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:56.377137 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:56.377347 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:56.377859 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 16:06:56.378376 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 16:06:56.384820 18959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:56.388626 18959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:56.410640 19695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.402009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41253 (local address 127.0.0.1:58760)
0806 16:06:56.403722 (+ 1713us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:56.403739 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:56.403914 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:56.404720 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:56.404724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:56.404752 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:56.405138 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.405148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.406697 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.406700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.408465 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.408478 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.410001 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.410005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.410127 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.410161 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 16:06:56.410245 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 16:06:56.410380 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1639,"mutex_wait_us":470,"thread_start_us":222,"threads_started":1}
I20260806 16:06:56.411201 19698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.403280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52922 (local address 127.8.49.2:37511)
0806 16:06:56.403852 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:56.403867 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:56.404264 (+ 397us) server_negotiation.cc:407] Connection header received
0806 16:06:56.404419 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:56.404423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:56.404594 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:56.404783 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:56.405297 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.406568 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.408641 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.409800 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.409914 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.410164 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:56.410718 (+ 554us) server_negotiation.cc:299] Negotiation successful
0806 16:06:56.410954 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":148,"threads_started":1}
I20260806 16:06:56.411340 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.402392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59790 (local address 127.8.49.3:35131)
0806 16:06:56.403183 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:56.403199 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:56.403771 (+ 572us) server_negotiation.cc:407] Connection header received
0806 16:06:56.403952 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:56.403962 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:56.404243 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:56.404417 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:56.404963 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.405915 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.408105 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.409261 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.409403 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.409642 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:56.410511 (+ 869us) server_negotiation.cc:299] Negotiation successful
0806 16:06:56.410656 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":167,"threads_started":1}
I20260806 16:06:56.411795 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.402182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:37511 (local address 127.0.0.1:52922)
0806 16:06:56.404134 (+ 1952us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:56.404149 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:56.404259 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:56.404883 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:56.404886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:56.404912 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:56.405102 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.405110 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.406701 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.406703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.408464 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.408478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.410318 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.410322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.410478 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.410501 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:06:56.410627 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 16:06:56.411629 (+ 1002us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1877,"mutex_wait_us":100,"thread_start_us":115,"threads_started":1}
I20260806 16:06:56.412133 19693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.401986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:35131 (local address 127.0.0.1:59790)
0806 16:06:56.403336 (+ 1350us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:56.403402 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:56.403585 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:56.404449 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:56.404455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:56.404491 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:56.404661 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.404673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.406063 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.406078 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.407901 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.407920 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.409734 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.409742 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.409906 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.409970 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 16:06:56.410138 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 16:06:56.410350 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"mutex_wait_us":4,"thread_start_us":209,"threads_started":1}
I20260806 16:06:56.412744 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.403399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58760 (local address 127.8.49.1:41253)
0806 16:06:56.404048 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:56.404054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:56.404081 (+ 27us) server_negotiation.cc:407] Connection header received
0806 16:06:56.404194 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:56.404199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:56.404480 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:56.404996 (+ 516us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:56.405316 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.406526 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.408641 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.409576 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.409716 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.409922 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:56.412411 (+ 2489us) server_negotiation.cc:299] Negotiation successful
0806 16:06:56.412580 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":129,"threads_started":1}
I20260806 16:06:56.416281 19570 tablet_service.cc:1467] Processing CreateTablet for tablet e4b010ed160548d895bb54c9fa1053be (DEFAULT_TABLE table=test-table [id=c0df53d945214967a208ecd6dc4a674f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:56.416285 19367 tablet_service.cc:1467] Processing CreateTablet for tablet e4b010ed160548d895bb54c9fa1053be (DEFAULT_TABLE table=test-table [id=c0df53d945214967a208ecd6dc4a674f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:56.417771 19367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b010ed160548d895bb54c9fa1053be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:56.417794 19570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b010ed160548d895bb54c9fa1053be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:56.418550 19168 tablet_service.cc:1467] Processing CreateTablet for tablet e4b010ed160548d895bb54c9fa1053be (DEFAULT_TABLE table=test-table [id=c0df53d945214967a208ecd6dc4a674f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:56.420172 19168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b010ed160548d895bb54c9fa1053be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:56.428969 19701 tablet_bootstrap.cc:492] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: Bootstrap starting.
I20260806 16:06:56.429913 19700 tablet_bootstrap.cc:492] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: Bootstrap starting.
I20260806 16:06:56.431897 19702 tablet_bootstrap.cc:492] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9: Bootstrap starting.
I20260806 16:06:56.432193 19701 tablet_bootstrap.cc:654] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:56.432257 19700 tablet_bootstrap.cc:654] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:56.433255 19701 log.cc:824] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:56.433255 19700 log.cc:824] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:56.435078 19702 tablet_bootstrap.cc:654] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:56.435907 19701 tablet_bootstrap.cc:492] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: No bootstrap required, opened a new log
I20260806 16:06:56.435935 19700 tablet_bootstrap.cc:492] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: No bootstrap required, opened a new log
I20260806 16:06:56.436161 19701 ts_tablet_manager.cc:1397] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 16:06:56.436161 19700 ts_tablet_manager.cc:1397] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 16:06:56.436265 19702 log.cc:824] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9: Log is configured to *not* fsync() on all Append() calls
I20260806 16:06:56.439407 19702 tablet_bootstrap.cc:492] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9: No bootstrap required, opened a new log
I20260806 16:06:56.439733 19702 ts_tablet_manager.cc:1397] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260806 16:06:56.441321 19701 raft_consensus.cc:348] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } }
I20260806 16:06:56.441303 19700 raft_consensus.cc:348] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } }
I20260806 16:06:56.441730 19701 raft_consensus.cc:374] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:56.441730 19700 raft_consensus.cc:374] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:56.441810 19701 raft_consensus.cc:729] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ed5b21cd654ed281efae740a991a75, State: Initialized, Role: FOLLOWER
I20260806 16:06:56.441810 19700 raft_consensus.cc:729] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 455f7250bda648c590c6c999665d81d5, State: Initialized, Role: FOLLOWER
I20260806 16:06:56.442428 19700 consensus_queue.cc:260] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } }
I20260806 16:06:56.442428 19701 consensus_queue.cc:260] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } }
I20260806 16:06:56.443789 19660 heartbeater.cc:503] Master 127.8.49.62:42755 was elected leader, sending a full tablet report...
I20260806 16:06:56.443789 19465 heartbeater.cc:503] Master 127.8.49.62:42755 was elected leader, sending a full tablet report...
I20260806 16:06:56.443872 19701 ts_tablet_manager.cc:1428] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20260806 16:06:56.443872 19700 ts_tablet_manager.cc:1428] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260806 16:06:56.445685 19702 raft_consensus.cc:348] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } }
I20260806 16:06:56.446086 19702 raft_consensus.cc:374] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:56.446300 19702 raft_consensus.cc:729] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5f2f647cde43dc908cacc2885682a9, State: Initialized, Role: FOLLOWER
I20260806 16:06:56.447057 19702 consensus_queue.cc:260] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } }
I20260806 16:06:56.447933 19260 heartbeater.cc:503] Master 127.8.49.62:42755 was elected leader, sending a full tablet report...
I20260806 16:06:56.448525 19702 ts_tablet_manager.cc:1428] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9: Time spent starting tablet: real 0.009s user 0.006s sys 0.002s
W20260806 16:06:56.474110 19466 tablet.cc:2369] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:56.546279 19711 raft_consensus.cc:482] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:56.546685 19711 raft_consensus.cc:504] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } }
I20260806 16:06:56.548602 19711 leader_election.cc:288] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83ed5b21cd654ed281efae740a991a75 (127.8.49.3:35131), 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511)
I20260806 16:06:56.554168 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.549253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:41727 (local address 127.8.49.3:35131)
0806 16:06:56.549491 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:56.549498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:56.549675 (+ 177us) server_negotiation.cc:407] Connection header received
0806 16:06:56.549960 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:56.549963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:56.550016 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:56.550111 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:56.550486 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.551356 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.552949 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.553380 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.553522 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.553601 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:56.553914 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 16:06:56.554004 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 16:06:56.554302 19713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.549196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:37511 (local address 127.8.49.1:43335)
0806 16:06:56.549743 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:56.549758 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:56.549933 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:56.550287 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:56.550290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:56.550311 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:56.550520 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.550527 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.551587 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.551592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.552796 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.552810 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.553579 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.553583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.553791 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.553815 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:06:56.553918 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 16:06:56.554116 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":201,"thread_start_us":226,"threads_started":1}
I20260806 16:06:56.554275 19698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.549486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:43335 (local address 127.8.49.2:37511)
0806 16:06:56.549627 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:56.549633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:56.549866 (+ 233us) server_negotiation.cc:407] Connection header received
0806 16:06:56.550090 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:56.550093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:56.550156 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:56.550262 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:56.550759 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.551437 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.552966 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.553422 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.553572 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.553673 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:56.553940 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 16:06:56.554081 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 16:06:56.554256 19712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.548929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:35131 (local address 127.8.49.1:41727)
0806 16:06:56.549548 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:56.549567 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:56.549804 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:56.550147 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:56.550151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:56.550174 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:56.550355 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.550362 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.551609 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.551614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.552784 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:56.552796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.553550 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.553553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:56.553685 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.553779 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 16:06:56.553881 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 16:06:56.554060 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":99,"thread_start_us":211,"threads_started":1}
I20260806 16:06:56.555261 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b010ed160548d895bb54c9fa1053be" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455f7250bda648c590c6c999665d81d5" is_pre_election: true
I20260806 16:06:56.555807 19598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b010ed160548d895bb54c9fa1053be" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ed5b21cd654ed281efae740a991a75" is_pre_election: true
I20260806 16:06:56.555946 19394 raft_consensus.cc:2454] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 0.
I20260806 16:06:56.556149 19598 raft_consensus.cc:2454] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 0.
I20260806 16:06:56.556841 19094 leader_election.cc:302] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ed5b21cd654ed281efae740a991a75, eb5f2f647cde43dc908cacc2885682a9; no voters:
I20260806 16:06:56.557215 19711 raft_consensus.cc:2781] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:56.557324 19711 raft_consensus.cc:482] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:56.557371 19711 raft_consensus.cc:3037] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:56.558806 19711 raft_consensus.cc:504] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } }
I20260806 16:06:56.559358 19711 leader_election.cc:288] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 election: Requested vote from peers 83ed5b21cd654ed281efae740a991a75 (127.8.49.3:35131), 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511)
I20260806 16:06:56.560124 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b010ed160548d895bb54c9fa1053be" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455f7250bda648c590c6c999665d81d5"
I20260806 16:06:56.560124 19598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b010ed160548d895bb54c9fa1053be" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ed5b21cd654ed281efae740a991a75"
I20260806 16:06:56.560384 19394 raft_consensus.cc:3037] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:56.560384 19598 raft_consensus.cc:3037] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:56.562088 19598 raft_consensus.cc:2454] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 1.
I20260806 16:06:56.562088 19394 raft_consensus.cc:2454] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 1.
I20260806 16:06:56.562657 19094 leader_election.cc:302] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 455f7250bda648c590c6c999665d81d5, 83ed5b21cd654ed281efae740a991a75, eb5f2f647cde43dc908cacc2885682a9; no voters:
I20260806 16:06:56.563037 19711 raft_consensus.cc:2781] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:56.563459 19711 raft_consensus.cc:686] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 1 LEADER]: Becoming Leader. State: Replica: eb5f2f647cde43dc908cacc2885682a9, State: Running, Role: LEADER
I20260806 16:06:56.564059 19711 consensus_queue.cc:237] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } }
W20260806 16:06:56.566627 19661 tablet.cc:2369] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:56.569062 18959 catalog_manager.cc:5495] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "eb5f2f647cde43dc908cacc2885682a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } health_report { overall_health: UNKNOWN } } }
I20260806 16:06:56.587798 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:56.581670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58774 (local address 127.8.49.1:41253)
0806 16:06:56.581895 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:56.581899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:56.581919 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:06:56.582029 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:56.582032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:56.582102 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:56.582232 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:56.583165 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.583785 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.585131 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:56.585390 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:56.585532 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:56.586185 (+ 653us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:56.587139 (+ 954us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:56.587275 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:56.587577 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 16:06:56.587660 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 16:06:56.598356 19598 raft_consensus.cc:1261] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Refusing update from remote peer eb5f2f647cde43dc908cacc2885682a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:56.598356 19394 raft_consensus.cc:1261] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Refusing update from remote peer eb5f2f647cde43dc908cacc2885682a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:56.599900 19714 consensus_queue.cc:1035] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:56.600317 19711 consensus_queue.cc:1035] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:56.611472 19719 mvcc.cc:205] Tried to move back new op lower bound from 7315588778373849088 to 7315588778250719232. Current Snapshot: MvccSnapshot[applied={T|T < 7315588778373849088}]
I20260806 16:06:56.612649 19721 mvcc.cc:205] Tried to move back new op lower bound from 7315588778373849088 to 7315588778250719232. Current Snapshot: MvccSnapshot[applied={T|T < 7315588778373849088}]
I20260806 16:06:56.614655 19722 mvcc.cc:205] Tried to move back new op lower bound from 7315588778373849088 to 7315588778250719232. Current Snapshot: MvccSnapshot[applied={T|T < 7315588778373849088}]
I20260806 16:06:56.625788 18960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:56.626119 18960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:56.630098 18960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6: Sending DeleteTablet for 3 replicas of tablet e4b010ed160548d895bb54c9fa1053be
I20260806 16:06:56.631486 19367 tablet_service.cc:1514] Processing DeleteTablet for tablet e4b010ed160548d895bb54c9fa1053be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:56 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:52922
I20260806 16:06:56.631628 19168 tablet_service.cc:1514] Processing DeleteTablet for tablet e4b010ed160548d895bb54c9fa1053be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:56 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:58760
I20260806 16:06:56.631628 19570 tablet_service.cc:1514] Processing DeleteTablet for tablet e4b010ed160548d895bb54c9fa1053be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:56 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:59790
I20260806 16:06:56.632342 19733 tablet_replica.cc:330] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: stopping tablet replica
I20260806 16:06:56.632764 19733 raft_consensus.cc:2229] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:56.633040 19731 tablet_replica.cc:330] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: stopping tablet replica
I20260806 16:06:56.633160 19733 raft_consensus.cc:2258] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:56.633961 19731 raft_consensus.cc:2229] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:56.634044 18960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 16:06:56.634464 19731 raft_consensus.cc:2258] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 16:06:56.634797 18960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:56.635062 19732 tablet_replica.cc:330] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9: stopping tablet replica
I20260806 16:06:56.635471 19732 raft_consensus.cc:2229] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:56.636180 19732 raft_consensus.cc:2258] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:56.639714 19731 ts_tablet_manager.cc:1905] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:56.640041 19733 ts_tablet_manager.cc:1905] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:56.643508 19367 tablet_service.cc:1467] Processing CreateTablet for tablet 1611d025463248fd90ae82d1660ee48d (DEFAULT_TABLE table=test-table [id=8d5915df9a3f4f33a343eb3cd5625779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:56.643934 19367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1611d025463248fd90ae82d1660ee48d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:56.644296 19570 tablet_service.cc:1467] Processing CreateTablet for tablet 1611d025463248fd90ae82d1660ee48d (DEFAULT_TABLE table=test-table [id=8d5915df9a3f4f33a343eb3cd5625779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:56.644629 19570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1611d025463248fd90ae82d1660ee48d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:56.647306 19700 tablet_bootstrap.cc:492] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5: Bootstrap starting.
I20260806 16:06:56.647239 19168 tablet_service.cc:1467] Processing CreateTablet for tablet 1611d025463248fd90ae82d1660ee48d (DEFAULT_TABLE table=test-table [id=8d5915df9a3f4f33a343eb3cd5625779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:56.647583 19168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1611d025463248fd90ae82d1660ee48d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:56.648213 19733 ts_tablet_manager.cc:1918] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:56.648389 19733 log.cc:1196] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/e4b010ed160548d895bb54c9fa1053be
I20260806 16:06:56.648612 19700 tablet_bootstrap.cc:654] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:56.648824 19733 ts_tablet_manager.cc:1939] T e4b010ed160548d895bb54c9fa1053be P 83ed5b21cd654ed281efae740a991a75: Deleting consensus metadata
I20260806 16:06:56.651899 18943 catalog_manager.cc:4841] TS 83ed5b21cd654ed281efae740a991a75 (127.8.49.3:35131): tablet e4b010ed160548d895bb54c9fa1053be (table test-table [id=c0df53d945214967a208ecd6dc4a674f]) successfully deleted
I20260806 16:06:56.653764 19701 tablet_bootstrap.cc:492] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75: Bootstrap starting.
I20260806 16:06:56.655429 19701 tablet_bootstrap.cc:654] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:56.657213 19702 tablet_bootstrap.cc:492] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9: Bootstrap starting.
I20260806 16:06:56.658615 19702 tablet_bootstrap.cc:654] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:56.658875 19701 tablet_bootstrap.cc:492] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75: No bootstrap required, opened a new log
I20260806 16:06:56.659009 19701 ts_tablet_manager.cc:1397] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 16:06:56.659626 19701 raft_consensus.cc:348] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:56.659785 19701 raft_consensus.cc:374] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:56.659822 19701 raft_consensus.cc:729] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ed5b21cd654ed281efae740a991a75, State: Initialized, Role: FOLLOWER
I20260806 16:06:56.659982 19701 consensus_queue.cc:260] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:56.660279 19701 ts_tablet_manager.cc:1428] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:06:56.662190 19732 ts_tablet_manager.cc:1905] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:56.662345 19702 tablet_bootstrap.cc:492] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9: No bootstrap required, opened a new log
I20260806 16:06:56.662571 19702 ts_tablet_manager.cc:1397] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 16:06:56.663589 19702 raft_consensus.cc:348] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:56.663743 19702 raft_consensus.cc:374] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:56.663779 19702 raft_consensus.cc:729] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5f2f647cde43dc908cacc2885682a9, State: Initialized, Role: FOLLOWER
I20260806 16:06:56.663938 19702 consensus_queue.cc:260] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:56.664366 19702 ts_tablet_manager.cc:1428] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:56.665982 19700 tablet_bootstrap.cc:492] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5: No bootstrap required, opened a new log
I20260806 16:06:56.666302 19700 ts_tablet_manager.cc:1397] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.003s
I20260806 16:06:56.667153 19732 ts_tablet_manager.cc:1918] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:56.667311 19732 log.cc:1196] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/wal/wals/e4b010ed160548d895bb54c9fa1053be
I20260806 16:06:56.667117 19700 raft_consensus.cc:348] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:56.667502 19700 raft_consensus.cc:374] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:56.667593 19700 raft_consensus.cc:729] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 455f7250bda648c590c6c999665d81d5, State: Initialized, Role: FOLLOWER
I20260806 16:06:56.667848 19700 consensus_queue.cc:260] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:56.668212 19700 ts_tablet_manager.cc:1428] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:56.669780 19731 ts_tablet_manager.cc:1918] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:56.669942 19731 log.cc:1196] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/e4b010ed160548d895bb54c9fa1053be
I20260806 16:06:56.675333 19731 ts_tablet_manager.cc:1939] T e4b010ed160548d895bb54c9fa1053be P 455f7250bda648c590c6c999665d81d5: Deleting consensus metadata
I20260806 16:06:56.675693 19732 ts_tablet_manager.cc:1939] T e4b010ed160548d895bb54c9fa1053be P eb5f2f647cde43dc908cacc2885682a9: Deleting consensus metadata
I20260806 16:06:56.677806 18940 catalog_manager.cc:4841] TS eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1:41253): tablet e4b010ed160548d895bb54c9fa1053be (table test-table [id=c0df53d945214967a208ecd6dc4a674f]) successfully deleted
I20260806 16:06:56.679064 18942 catalog_manager.cc:4841] TS 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511): tablet e4b010ed160548d895bb54c9fa1053be (table test-table [id=c0df53d945214967a208ecd6dc4a674f]) successfully deleted
W20260806 16:06:56.900357 19264 tablet.cc:2369] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:06:56.948230 19714 raft_consensus.cc:482] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:56.948514 19714 raft_consensus.cc:504] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:56.949278 19714 leader_election.cc:288] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511), 83ed5b21cd654ed281efae740a991a75 (127.8.49.3:35131)
I20260806 16:06:56.949925 19598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1611d025463248fd90ae82d1660ee48d" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ed5b21cd654ed281efae740a991a75" is_pre_election: true
I20260806 16:06:56.950052 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1611d025463248fd90ae82d1660ee48d" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455f7250bda648c590c6c999665d81d5" is_pre_election: true
I20260806 16:06:56.950176 19598 raft_consensus.cc:2454] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 0.
I20260806 16:06:56.950333 19394 raft_consensus.cc:2454] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 0.
I20260806 16:06:56.950593 19094 leader_election.cc:302] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ed5b21cd654ed281efae740a991a75, eb5f2f647cde43dc908cacc2885682a9; no voters:
I20260806 16:06:56.950892 19714 raft_consensus.cc:2781] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:56.950975 19714 raft_consensus.cc:482] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:56.951009 19714 raft_consensus.cc:3037] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:56.952482 19714 raft_consensus.cc:504] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:56.953042 19714 leader_election.cc:288] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 election: Requested vote from peers 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511), 83ed5b21cd654ed281efae740a991a75 (127.8.49.3:35131)
I20260806 16:06:56.953689 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1611d025463248fd90ae82d1660ee48d" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455f7250bda648c590c6c999665d81d5"
I20260806 16:06:56.953886 19394 raft_consensus.cc:3037] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:56.953987 19598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1611d025463248fd90ae82d1660ee48d" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ed5b21cd654ed281efae740a991a75"
I20260806 16:06:56.954216 19598 raft_consensus.cc:3037] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:56.955453 19394 raft_consensus.cc:2454] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 1.
I20260806 16:06:56.955894 19598 raft_consensus.cc:2454] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 1.
I20260806 16:06:56.955931 19093 leader_election.cc:302] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 455f7250bda648c590c6c999665d81d5, eb5f2f647cde43dc908cacc2885682a9; no voters:
I20260806 16:06:56.956257 19714 raft_consensus.cc:2781] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:56.956388 19714 raft_consensus.cc:686] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 1 LEADER]: Becoming Leader. State: Replica: eb5f2f647cde43dc908cacc2885682a9, State: Running, Role: LEADER
I20260806 16:06:56.956611 19714 consensus_queue.cc:237] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:56.959239 18960 catalog_manager.cc:5495] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "eb5f2f647cde43dc908cacc2885682a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } health_report { overall_health: UNKNOWN } } }
Aug 06 16:06:57 dist-test-slave-7z32 krb5kdc[18893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032415, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.62@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
I20260806 16:06:57.053433 19751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.041282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35648 (local address 127.8.49.62:42755)
0806 16:06:57.041707 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:57.041716 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:57.041739 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:06:57.041903 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:57.041906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:57.041975 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:57.042095 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:57.043234 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.043842 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.045023 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.045262 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.045425 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.048966 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:57.048970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:57.048973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:57.049004 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:57.050778 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:57.051609 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:57.051612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:57.051614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:57.051709 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:57.052106 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:57.052109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:57.052110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:57.052323 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:57.052412 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:57.052991 (+ 579us) server_negotiation.cc:299] Negotiation successful
0806 16:06:57.053256 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":331,"thread_start_us":181,"threads_started":1}
I20260806 16:06:57.053550 19750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.041054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42755 (local address 127.0.0.1:35648)
0806 16:06:57.041503 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:57.041527 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:57.041726 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:57.042230 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:57.042233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:57.042791 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:06:57.043010 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.043036 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.044009 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.044013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.044801 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.044812 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.045444 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.045448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.045534 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.046419 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:06:57.046434 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:06:57.048745 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:06:57.050997 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:57.051015 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:57.051034 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:57.051462 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:57.051859 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:06:57.051862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:06:57.051864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:06:57.051974 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:06:57.052460 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:06:57.052469 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:06:57.052733 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 16:06:57.052897 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 16:06:57.053197 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":103,"threads_started":1}
I20260806 16:06:57.059724 18960 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:35648:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 16:06:57.061129 18960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:57.069423 19168 tablet_service.cc:1467] Processing CreateTablet for tablet cdf9e1c67e25438ab90a115f622916a9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f03e413fb1b44bcfb5af96d24a3f36d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:06:57.069504 19570 tablet_service.cc:1467] Processing CreateTablet for tablet cdf9e1c67e25438ab90a115f622916a9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f03e413fb1b44bcfb5af96d24a3f36d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:06:57.070031 19570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdf9e1c67e25438ab90a115f622916a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:57.070087 19168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdf9e1c67e25438ab90a115f622916a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:57.072762 19367 tablet_service.cc:1467] Processing CreateTablet for tablet cdf9e1c67e25438ab90a115f622916a9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f03e413fb1b44bcfb5af96d24a3f36d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 16:06:57.073297 19367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdf9e1c67e25438ab90a115f622916a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:57.073724 19702 tablet_bootstrap.cc:492] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9: Bootstrap starting.
I20260806 16:06:57.073894 19701 tablet_bootstrap.cc:492] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: Bootstrap starting.
I20260806 16:06:57.075583 19702 tablet_bootstrap.cc:654] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:57.075619 19701 tablet_bootstrap.cc:654] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:57.077191 19702 tablet_bootstrap.cc:492] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9: No bootstrap required, opened a new log
I20260806 16:06:57.077323 19702 ts_tablet_manager.cc:1397] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:57.077574 19700 tablet_bootstrap.cc:492] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5: Bootstrap starting.
I20260806 16:06:57.077631 19701 tablet_bootstrap.cc:492] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: No bootstrap required, opened a new log
I20260806 16:06:57.077812 19701 ts_tablet_manager.cc:1397] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 16:06:57.078084 19702 raft_consensus.cc:348] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.078244 19702 raft_consensus.cc:374] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:57.078285 19702 raft_consensus.cc:729] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5f2f647cde43dc908cacc2885682a9, State: Initialized, Role: FOLLOWER
I20260806 16:06:57.078317 19701 raft_consensus.cc:348] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.078474 19701 raft_consensus.cc:374] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:57.078524 19701 raft_consensus.cc:729] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ed5b21cd654ed281efae740a991a75, State: Initialized, Role: FOLLOWER
I20260806 16:06:57.078510 19702 consensus_queue.cc:260] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.078666 19701 consensus_queue.cc:260] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.078996 19700 tablet_bootstrap.cc:654] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:57.078969 19714 tablet_replica.cc:439] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } }
I20260806 16:06:57.079203 19714 tablet_replica.cc:442] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:57.079409 19702 ts_tablet_manager.cc:1428] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:57.079708 19701 ts_tablet_manager.cc:1428] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:57.079631 19709 tablet_replica.cc:439] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } }
I20260806 16:06:57.079810 19709 tablet_replica.cc:442] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:57.081138 19700 tablet_bootstrap.cc:492] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5: No bootstrap required, opened a new log
I20260806 16:06:57.081255 19700 ts_tablet_manager.cc:1397] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:57.082113 19700 raft_consensus.cc:348] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.082278 19700 raft_consensus.cc:374] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:57.082314 19700 raft_consensus.cc:729] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 455f7250bda648c590c6c999665d81d5, State: Initialized, Role: FOLLOWER
I20260806 16:06:57.082495 19700 consensus_queue.cc:260] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.083041 19708 tablet_replica.cc:439] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } }
I20260806 16:06:57.083324 19708 tablet_replica.cc:442] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:57.083577 19700 ts_tablet_manager.cc:1428] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 16:06:57.144266 19709 raft_consensus.cc:482] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:57.144549 19709 raft_consensus.cc:504] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.146173 19709 leader_election.cc:288] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1:41253), 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511)
I20260806 16:06:57.153407 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.146900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:48603 (local address 127.8.49.1:41253)
0806 16:06:57.147314 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:57.147322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:57.147342 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:06:57.147430 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:57.147434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:57.147505 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:57.147669 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:57.148314 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.149279 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.151638 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.152439 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.152630 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.152750 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:57.153070 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 16:06:57.153256 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":115,"threads_started":1}
I20260806 16:06:57.153378 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.146626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41253 (local address 127.8.49.3:48603)
0806 16:06:57.147138 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:57.147157 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:57.147289 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:57.147717 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:57.147721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:57.147773 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:57.148088 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.148097 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.149514 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.149520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.151415 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.151433 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.152694 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.152707 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.152813 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.152856 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 16:06:57.153020 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 16:06:57.153135 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":150,"spinlock_wait_cycles":41088,"thread_start_us":120,"threads_started":1}
I20260806 16:06:57.153548 19761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.146902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:42707 (local address 127.8.49.2:37511)
0806 16:06:57.147265 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:57.147271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:57.147298 (+ 27us) server_negotiation.cc:407] Connection header received
0806 16:06:57.147430 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:57.147434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:57.147505 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:57.147676 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:06:57.148314 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.149279 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.151863 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.152573 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.152703 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.152776 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:57.153170 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 16:06:57.153275 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":131,"threads_started":1}
I20260806 16:06:57.154202 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.146626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:37511 (local address 127.8.49.3:42707)
0806 16:06:57.147223 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:57.147236 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:57.147298 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:57.147717 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:57.147721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:57.147774 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:06:57.148088 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.148097 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.149547 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.149550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.151415 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.151433 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.152881 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.152885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.152996 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.153022 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:06:57.153150 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 16:06:57.153273 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":295,"spinlock_wait_cycles":11264,"thread_start_us":69,"threads_started":1}
I20260806 16:06:57.154409 19195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf9e1c67e25438ab90a115f622916a9" candidate_uuid: "83ed5b21cd654ed281efae740a991a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5f2f647cde43dc908cacc2885682a9" is_pre_election: true
I20260806 16:06:57.154788 19195 raft_consensus.cc:2454] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ed5b21cd654ed281efae740a991a75 in term 0.
I20260806 16:06:57.154974 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf9e1c67e25438ab90a115f622916a9" candidate_uuid: "83ed5b21cd654ed281efae740a991a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455f7250bda648c590c6c999665d81d5" is_pre_election: true
I20260806 16:06:57.155184 19394 raft_consensus.cc:2454] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ed5b21cd654ed281efae740a991a75 in term 0.
I20260806 16:06:57.155522 19497 leader_election.cc:302] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ed5b21cd654ed281efae740a991a75, eb5f2f647cde43dc908cacc2885682a9; no voters:
I20260806 16:06:57.155932 19709 raft_consensus.cc:2781] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:57.156016 19709 raft_consensus.cc:482] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:57.156055 19709 raft_consensus.cc:3037] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:57.157392 19709 raft_consensus.cc:504] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.157855 19709 leader_election.cc:288] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [CANDIDATE]: Term 1 election: Requested vote from peers eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1:41253), 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511)
I20260806 16:06:57.158389 19195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf9e1c67e25438ab90a115f622916a9" candidate_uuid: "83ed5b21cd654ed281efae740a991a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5f2f647cde43dc908cacc2885682a9"
I20260806 16:06:57.158432 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdf9e1c67e25438ab90a115f622916a9" candidate_uuid: "83ed5b21cd654ed281efae740a991a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455f7250bda648c590c6c999665d81d5"
I20260806 16:06:57.158552 19195 raft_consensus.cc:3037] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:57.158600 19394 raft_consensus.cc:3037] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:57.159881 19394 raft_consensus.cc:2454] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ed5b21cd654ed281efae740a991a75 in term 1.
I20260806 16:06:57.160113 19195 raft_consensus.cc:2454] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ed5b21cd654ed281efae740a991a75 in term 1.
I20260806 16:06:57.160357 19499 leader_election.cc:302] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 455f7250bda648c590c6c999665d81d5, 83ed5b21cd654ed281efae740a991a75; no voters:
I20260806 16:06:57.160597 19709 raft_consensus.cc:2781] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:57.160869 19709 raft_consensus.cc:686] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [term 1 LEADER]: Becoming Leader. State: Replica: 83ed5b21cd654ed281efae740a991a75, State: Running, Role: LEADER
I20260806 16:06:57.161228 19709 consensus_queue.cc:237] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.162804 19763 tablet_replica.cc:439] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: TxnStatusTablet state changed. Reason: New leader 83ed5b21cd654ed281efae740a991a75. Latest consensus state: current_term: 1 leader_uuid: "83ed5b21cd654ed281efae740a991a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } }
I20260806 16:06:57.163005 19763 tablet_replica.cc:442] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:06:57.163326 19765 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 16:06:57.163527 19765 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 16:06:57.165107 18955 catalog_manager.cc:5495] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ed5b21cd654ed281efae740a991a75 (127.8.49.3). New cstate: current_term: 1 leader_uuid: "83ed5b21cd654ed281efae740a991a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } health_report { overall_health: HEALTHY } } }
I20260806 16:06:57.190212 19750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.184755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:35131 (local address 127.0.0.1:59792)
0806 16:06:57.184998 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:57.185018 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:57.185272 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:57.185812 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:57.185815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:57.185828 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:06:57.185967 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.185973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.186902 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.186906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.187591 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.187600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.188115 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.188119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.188218 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.188280 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:06:57.189823 (+ 1543us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:06:57.189844 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 16:06:57.189921 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 16:06:57.190049 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 16:06:57.190230 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.185010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.8.49.3:35131)
0806 16:06:57.185439 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:57.185450 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:57.185471 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:06:57.185567 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:57.185570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:57.185651 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:57.185784 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:57.186085 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.186727 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.187729 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.187983 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.188102 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.189036 (+ 934us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:57.189708 (+ 672us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:57.189856 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:57.189931 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 16:06:57.190004 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":173,"threads_started":1}
I20260806 16:06:57.194924 19394 raft_consensus.cc:1261] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Refusing update from remote peer 83ed5b21cd654ed281efae740a991a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:57.195185 19195 raft_consensus.cc:1261] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Refusing update from remote peer 83ed5b21cd654ed281efae740a991a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:57.195623 19709 consensus_queue.cc:1035] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:57.196534 19709 consensus_queue.cc:1035] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:57.198443 19708 tablet_replica.cc:439] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5: TxnStatusTablet state changed. Reason: New leader 83ed5b21cd654ed281efae740a991a75. Latest consensus state: current_term: 1 leader_uuid: "83ed5b21cd654ed281efae740a991a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } }
I20260806 16:06:57.198614 19708 tablet_replica.cc:442] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:57.199342 19714 tablet_replica.cc:439] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9: TxnStatusTablet state changed. Reason: New leader 83ed5b21cd654ed281efae740a991a75. Latest consensus state: current_term: 1 leader_uuid: "83ed5b21cd654ed281efae740a991a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } }
I20260806 16:06:57.199532 19714 tablet_replica.cc:442] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:57.202653 19709 tablet_replica.cc:439] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83ed5b21cd654ed281efae740a991a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } }
I20260806 16:06:57.202801 19709 tablet_replica.cc:442] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:06:57.203385 19714 tablet_replica.cc:439] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83ed5b21cd654ed281efae740a991a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } }
I20260806 16:06:57.203563 19714 tablet_replica.cc:442] T cdf9e1c67e25438ab90a115f622916a9 P eb5f2f647cde43dc908cacc2885682a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:57.207079 19709 tablet_replica.cc:439] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83ed5b21cd654ed281efae740a991a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } }
I20260806 16:06:57.207243 19709 tablet_replica.cc:442] T cdf9e1c67e25438ab90a115f622916a9 P 83ed5b21cd654ed281efae740a991a75: This TxnStatusTablet replica's current role is: LEADER
I20260806 16:06:57.208841 19708 tablet_replica.cc:439] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83ed5b21cd654ed281efae740a991a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } } }
I20260806 16:06:57.209049 19708 tablet_replica.cc:442] T cdf9e1c67e25438ab90a115f622916a9 P 455f7250bda648c590c6c999665d81d5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 16:06:57.228144 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.222429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:52941 (local address 127.8.49.3:35131)
0806 16:06:57.222612 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:57.222616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:57.222970 (+ 354us) server_negotiation.cc:407] Connection header received
0806 16:06:57.223179 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:57.223183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:57.223291 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:57.223429 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:57.223736 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.224680 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.226321 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.226687 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.226827 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.227530 (+ 703us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:57.227659 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:57.227754 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:57.227885 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 16:06:57.227949 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 16:06:57.228147 19788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.222268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:35131 (local address 127.8.49.1:52941)
0806 16:06:57.222786 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:57.222800 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:57.223002 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:57.223464 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:57.223467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:57.223475 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:06:57.223596 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.223601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.224923 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.224932 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.226072 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.226085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.226867 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.226872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.226973 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.227219 (+ 246us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:06:57.227761 (+ 542us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:06:57.227771 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 16:06:57.227870 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 16:06:57.227971 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":131,"threads_started":1}
I20260806 16:06:57.242798 19598 raft_consensus.cc:1261] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Refusing update from remote peer eb5f2f647cde43dc908cacc2885682a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:57.242920 19394 raft_consensus.cc:1261] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Refusing update from remote peer eb5f2f647cde43dc908cacc2885682a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:57.243643 19714 consensus_queue.cc:1035] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:57.243964 19711 consensus_queue.cc:1035] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:57.277909 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.271940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:39813 (local address 127.8.49.1:41253)
0806 16:06:57.272186 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:57.272192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:57.272404 (+ 212us) server_negotiation.cc:407] Connection header received
0806 16:06:57.272658 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:57.272662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:57.272745 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:57.272799 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:57.273334 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.274179 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.275716 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.276208 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.276313 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.277179 (+ 866us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:57.277283 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:57.277337 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:57.277619 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 16:06:57.277702 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:06:57.279088 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.271504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41253 (local address 127.8.49.3:39813)
0806 16:06:57.272301 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:57.272319 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:06:57.272517 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:06:57.273028 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:06:57.273033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:06:57.273047 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 16:06:57.273198 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.273204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.274364 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.274368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.275529 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:06:57.275546 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.276564 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.276567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:06:57.276709 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.276798 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 16:06:57.277408 (+ 610us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 16:06:57.277478 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 16:06:57.278422 (+ 944us) client_negotiation.cc:240] Negotiation successful
0806 16:06:57.278933 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":133,"threads_started":1}
I20260806 16:06:57.313076 18960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:57.313354 18960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:57.316695 18960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6: Sending DeleteTablet for 3 replicas of tablet 1611d025463248fd90ae82d1660ee48d
I20260806 16:06:57.317575 19570 tablet_service.cc:1514] Processing DeleteTablet for tablet 1611d025463248fd90ae82d1660ee48d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:57 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:59790
I20260806 16:06:57.317644 19168 tablet_service.cc:1514] Processing DeleteTablet for tablet 1611d025463248fd90ae82d1660ee48d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:57 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:58760
I20260806 16:06:57.317871 19367 tablet_service.cc:1514] Processing DeleteTablet for tablet 1611d025463248fd90ae82d1660ee48d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:57 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:52922
I20260806 16:06:57.322770 19800 tablet_replica.cc:330] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9: stopping tablet replica
I20260806 16:06:57.323058 19800 raft_consensus.cc:2229] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:57.323400 19800 raft_consensus.cc:2258] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:57.324465 19800 ts_tablet_manager.cc:1905] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:57.329654 19801 tablet_replica.cc:330] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75: stopping tablet replica
I20260806 16:06:57.329846 19801 raft_consensus.cc:2229] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:57.330022 19801 raft_consensus.cc:2258] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:57.331176 19801 ts_tablet_manager.cc:1905] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:57.332147 19800 ts_tablet_manager.cc:1918] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:57.332280 19800 log.cc:1196] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/wal/wals/1611d025463248fd90ae82d1660ee48d
I20260806 16:06:57.332731 19800 ts_tablet_manager.cc:1939] T 1611d025463248fd90ae82d1660ee48d P eb5f2f647cde43dc908cacc2885682a9: Deleting consensus metadata
I20260806 16:06:57.333885 18940 catalog_manager.cc:4841] TS eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1:41253): tablet 1611d025463248fd90ae82d1660ee48d (table test-table [id=8d5915df9a3f4f33a343eb3cd5625779]) successfully deleted
I20260806 16:06:57.335309 19801 ts_tablet_manager.cc:1918] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:57.335426 19801 log.cc:1196] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/1611d025463248fd90ae82d1660ee48d
I20260806 16:06:57.335733 19801 ts_tablet_manager.cc:1939] T 1611d025463248fd90ae82d1660ee48d P 83ed5b21cd654ed281efae740a991a75: Deleting consensus metadata
I20260806 16:06:57.336277 19802 tablet_replica.cc:330] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5: stopping tablet replica
I20260806 16:06:57.336483 19802 raft_consensus.cc:2229] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:57.336656 19802 raft_consensus.cc:2258] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:57.336848 18943 catalog_manager.cc:4841] TS 83ed5b21cd654ed281efae740a991a75 (127.8.49.3:35131): tablet 1611d025463248fd90ae82d1660ee48d (table test-table [id=8d5915df9a3f4f33a343eb3cd5625779]) successfully deleted
I20260806 16:06:57.338016 19802 ts_tablet_manager.cc:1905] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 16:06:57 dist-test-slave-7z32 krb5kdc[18893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260806 16:06:57.343007 19802 ts_tablet_manager.cc:1918] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:57.343245 19802 log.cc:1196] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/1611d025463248fd90ae82d1660ee48d
I20260806 16:06:57.343659 19802 ts_tablet_manager.cc:1939] T 1611d025463248fd90ae82d1660ee48d P 455f7250bda648c590c6c999665d81d5: Deleting consensus metadata
I20260806 16:06:57.344888 18942 catalog_manager.cc:4841] TS 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511): tablet 1611d025463248fd90ae82d1660ee48d (table test-table [id=8d5915df9a3f4f33a343eb3cd5625779]) successfully deleted
Aug 06 16:06:57 dist-test-slave-7z32 krb5kdc[18893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:06:57 dist-test-slave-7z32 krb5kdc[18893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:06:57.361371 19751 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:06:57.352704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35664 (local address 127.8.49.62:42755)
0806 16:06:57.352903 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:57.352908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:57.352937 (+ 29us) server_negotiation.cc:407] Connection header received
0806 16:06:57.353127 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:57.353131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:57.353207 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:57.353370 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:57.354391 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.355426 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.356719 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.357275 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.357436 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.361181 (+ 3745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35664: BlockingRecv error: recv got EOF from 127.0.0.1:35664 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 16:06:57 dist-test-slave-7z32 krb5kdc[18893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM
I20260806 16:06:57.374928 19751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.363042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35672 (local address 127.8.49.62:42755)
0806 16:06:57.363207 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:57.363212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:57.363239 (+ 27us) server_negotiation.cc:407] Connection header received
0806 16:06:57.363412 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:57.363416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:57.363487 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:57.363617 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:06:57.364443 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.365518 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.366626 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.367047 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.367172 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.369970 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:06:57.369975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:06:57.369978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:06:57.370021 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:06:57.372381 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:57.373038 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:57.373042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:57.373044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:57.373152 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:06:57.373491 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:06:57.373494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:06:57.373496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:06:57.373812 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:06:57.374005 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:57.374284 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 16:06:57.374667 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:06:57.380810 18960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:57.381507 18960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:57.389168 19168 tablet_service.cc:1467] Processing CreateTablet for tablet b436cacb1124421681e1b26622adc284 (DEFAULT_TABLE table=test-table [id=1058a8210086475c90d7c27537ea04f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:57.389235 19570 tablet_service.cc:1467] Processing CreateTablet for tablet b436cacb1124421681e1b26622adc284 (DEFAULT_TABLE table=test-table [id=1058a8210086475c90d7c27537ea04f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:57.389590 19168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b436cacb1124421681e1b26622adc284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:57.389588 19570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b436cacb1124421681e1b26622adc284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:57.389726 19367 tablet_service.cc:1467] Processing CreateTablet for tablet b436cacb1124421681e1b26622adc284 (DEFAULT_TABLE table=test-table [id=1058a8210086475c90d7c27537ea04f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:57.390177 19367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b436cacb1124421681e1b26622adc284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:57.392656 19702 tablet_bootstrap.cc:492] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9: Bootstrap starting.
I20260806 16:06:57.393007 19701 tablet_bootstrap.cc:492] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75: Bootstrap starting.
I20260806 16:06:57.393779 19702 tablet_bootstrap.cc:654] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:57.393925 19701 tablet_bootstrap.cc:654] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:57.394546 19700 tablet_bootstrap.cc:492] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5: Bootstrap starting.
I20260806 16:06:57.395857 19700 tablet_bootstrap.cc:654] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:57.396550 19701 tablet_bootstrap.cc:492] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75: No bootstrap required, opened a new log
I20260806 16:06:57.396704 19701 ts_tablet_manager.cc:1397] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:06:57.397442 19701 raft_consensus.cc:348] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.397604 19701 raft_consensus.cc:374] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:57.397594 19700 tablet_bootstrap.cc:492] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5: No bootstrap required, opened a new log
I20260806 16:06:57.397641 19701 raft_consensus.cc:729] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ed5b21cd654ed281efae740a991a75, State: Initialized, Role: FOLLOWER
I20260806 16:06:57.397709 19700 ts_tablet_manager.cc:1397] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260806 16:06:57.397792 19701 consensus_queue.cc:260] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.397928 19702 tablet_bootstrap.cc:492] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9: No bootstrap required, opened a new log
I20260806 16:06:57.398023 19702 ts_tablet_manager.cc:1397] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 16:06:57.398094 19701 ts_tablet_manager.cc:1428] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:06:57.398494 19702 raft_consensus.cc:348] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.398634 19702 raft_consensus.cc:374] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:57.398669 19702 raft_consensus.cc:729] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5f2f647cde43dc908cacc2885682a9, State: Initialized, Role: FOLLOWER
I20260806 16:06:57.398855 19702 consensus_queue.cc:260] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.399170 19702 ts_tablet_manager.cc:1428] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:06:57.399710 19700 raft_consensus.cc:348] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.399912 19700 raft_consensus.cc:374] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:57.399983 19700 raft_consensus.cc:729] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 455f7250bda648c590c6c999665d81d5, State: Initialized, Role: FOLLOWER
I20260806 16:06:57.400185 19700 consensus_queue.cc:260] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.400557 19700 ts_tablet_manager.cc:1428] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 16:06:57.488910 19763 raft_consensus.cc:482] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:57.489158 19763 raft_consensus.cc:504] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.489727 19763 leader_election.cc:288] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1:41253), 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511)
I20260806 16:06:57.490554 19195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b436cacb1124421681e1b26622adc284" candidate_uuid: "83ed5b21cd654ed281efae740a991a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5f2f647cde43dc908cacc2885682a9" is_pre_election: true
I20260806 16:06:57.490587 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b436cacb1124421681e1b26622adc284" candidate_uuid: "83ed5b21cd654ed281efae740a991a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455f7250bda648c590c6c999665d81d5" is_pre_election: true
I20260806 16:06:57.490821 19394 raft_consensus.cc:2454] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ed5b21cd654ed281efae740a991a75 in term 0.
I20260806 16:06:57.490792 19195 raft_consensus.cc:2454] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ed5b21cd654ed281efae740a991a75 in term 0.
I20260806 16:06:57.491341 19499 leader_election.cc:302] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 455f7250bda648c590c6c999665d81d5, 83ed5b21cd654ed281efae740a991a75; no voters:
I20260806 16:06:57.491587 19763 raft_consensus.cc:2781] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:57.491683 19763 raft_consensus.cc:482] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:57.491730 19763 raft_consensus.cc:3037] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:57.493199 19763 raft_consensus.cc:504] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.493685 19763 leader_election.cc:288] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [CANDIDATE]: Term 1 election: Requested vote from peers eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1:41253), 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511)
I20260806 16:06:57.494240 19195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b436cacb1124421681e1b26622adc284" candidate_uuid: "83ed5b21cd654ed281efae740a991a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5f2f647cde43dc908cacc2885682a9"
I20260806 16:06:57.494266 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b436cacb1124421681e1b26622adc284" candidate_uuid: "83ed5b21cd654ed281efae740a991a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455f7250bda648c590c6c999665d81d5"
I20260806 16:06:57.494393 19195 raft_consensus.cc:3037] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:57.494541 19394 raft_consensus.cc:3037] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:57.495777 19195 raft_consensus.cc:2454] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ed5b21cd654ed281efae740a991a75 in term 1.
I20260806 16:06:57.495975 19394 raft_consensus.cc:2454] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ed5b21cd654ed281efae740a991a75 in term 1.
I20260806 16:06:57.496280 19497 leader_election.cc:302] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ed5b21cd654ed281efae740a991a75, eb5f2f647cde43dc908cacc2885682a9; no voters:
I20260806 16:06:57.496491 19763 raft_consensus.cc:2781] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:57.496626 19763 raft_consensus.cc:686] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 1 LEADER]: Becoming Leader. State: Replica: 83ed5b21cd654ed281efae740a991a75, State: Running, Role: LEADER
I20260806 16:06:57.496830 19763 consensus_queue.cc:237] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.499962 18960 catalog_manager.cc:5495] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ed5b21cd654ed281efae740a991a75 (127.8.49.3). New cstate: current_term: 1 leader_uuid: "83ed5b21cd654ed281efae740a991a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } health_report { overall_health: HEALTHY } } }
I20260806 16:06:57.513944 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.509295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59794 (local address 127.8.49.3:35131)
0806 16:06:57.509466 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:57.509470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:57.509490 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:06:57.509632 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:57.509634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:57.509692 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:57.509777 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:57.510329 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.511025 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.512165 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.512465 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.512595 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.513234 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:57.513332 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:57.513477 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:57.513723 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 16:06:57.513795 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:06:57.517897 19394 raft_consensus.cc:1261] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Refusing update from remote peer 83ed5b21cd654ed281efae740a991a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:57.518112 19195 raft_consensus.cc:1261] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Refusing update from remote peer 83ed5b21cd654ed281efae740a991a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:57.518574 19763 consensus_queue.cc:1035] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:57.518887 19709 consensus_queue.cc:1035] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:57.534037 18955 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:57.534315 18955 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:57.537708 18955 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6: Sending DeleteTablet for 3 replicas of tablet b436cacb1124421681e1b26622adc284
I20260806 16:06:57.538918 19367 tablet_service.cc:1514] Processing DeleteTablet for tablet b436cacb1124421681e1b26622adc284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:57 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:52922
I20260806 16:06:57.538995 19168 tablet_service.cc:1514] Processing DeleteTablet for tablet b436cacb1124421681e1b26622adc284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:57 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:58760
I20260806 16:06:57.539230 19800 tablet_replica.cc:330] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9: stopping tablet replica
I20260806 16:06:57.539250 19802 tablet_replica.cc:330] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5: stopping tablet replica
I20260806 16:06:57.539445 19800 raft_consensus.cc:2229] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:57.539462 19802 raft_consensus.cc:2229] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:57.539654 19800 raft_consensus.cc:2258] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:57.539681 19802 raft_consensus.cc:2258] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:57.539981 19570 tablet_service.cc:1514] Processing DeleteTablet for tablet b436cacb1124421681e1b26622adc284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:57 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:59790
I20260806 16:06:57.540218 19801 tablet_replica.cc:330] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75: stopping tablet replica
I20260806 16:06:57.540378 19801 raft_consensus.cc:2229] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:57.540817 19801 raft_consensus.cc:2258] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:57.540899 19802 ts_tablet_manager.cc:1905] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:57.542286 19800 ts_tablet_manager.cc:1905] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:57.544791 19802 ts_tablet_manager.cc:1918] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:57.544929 19802 log.cc:1196] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/b436cacb1124421681e1b26622adc284
I20260806 16:06:57.545296 19800 ts_tablet_manager.cc:1918] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:57.545450 19800 log.cc:1196] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-0/wal/wals/b436cacb1124421681e1b26622adc284
I20260806 16:06:57.545588 19802 ts_tablet_manager.cc:1939] T b436cacb1124421681e1b26622adc284 P 455f7250bda648c590c6c999665d81d5: Deleting consensus metadata
I20260806 16:06:57.545881 19800 ts_tablet_manager.cc:1939] T b436cacb1124421681e1b26622adc284 P eb5f2f647cde43dc908cacc2885682a9: Deleting consensus metadata
I20260806 16:06:57.547145 18940 catalog_manager.cc:4841] TS eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1:41253): tablet b436cacb1124421681e1b26622adc284 (table test-table [id=1058a8210086475c90d7c27537ea04f3]) successfully deleted
I20260806 16:06:57.547489 19801 ts_tablet_manager.cc:1905] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:57.548441 18942 catalog_manager.cc:4841] TS 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511): tablet b436cacb1124421681e1b26622adc284 (table test-table [id=1058a8210086475c90d7c27537ea04f3]) successfully deleted
I20260806 16:06:57.549281 18955 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:06:57.549907 18955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:06:57.550796 19801 ts_tablet_manager.cc:1918] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:06:57.550979 19801 log.cc:1196] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/b436cacb1124421681e1b26622adc284
I20260806 16:06:57.551342 19801 ts_tablet_manager.cc:1939] T b436cacb1124421681e1b26622adc284 P 83ed5b21cd654ed281efae740a991a75: Deleting consensus metadata
I20260806 16:06:57.552343 18943 catalog_manager.cc:4841] TS 83ed5b21cd654ed281efae740a991a75 (127.8.49.3:35131): tablet b436cacb1124421681e1b26622adc284 (table test-table [id=1058a8210086475c90d7c27537ea04f3]) successfully deleted
I20260806 16:06:57.556965 19168 tablet_service.cc:1467] Processing CreateTablet for tablet ffd81796639046b992db03e7a6749771 (DEFAULT_TABLE table=test-table [id=6330439223154eea9c3976804870acab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:57.557040 19367 tablet_service.cc:1467] Processing CreateTablet for tablet ffd81796639046b992db03e7a6749771 (DEFAULT_TABLE table=test-table [id=6330439223154eea9c3976804870acab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:57.557348 19168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd81796639046b992db03e7a6749771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:57.557363 19367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd81796639046b992db03e7a6749771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:57.560182 19702 tablet_bootstrap.cc:492] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9: Bootstrap starting.
I20260806 16:06:57.560678 19700 tablet_bootstrap.cc:492] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5: Bootstrap starting.
I20260806 16:06:57.561271 19702 tablet_bootstrap.cc:654] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:57.562048 19570 tablet_service.cc:1467] Processing CreateTablet for tablet ffd81796639046b992db03e7a6749771 (DEFAULT_TABLE table=test-table [id=6330439223154eea9c3976804870acab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:06:57.562359 19570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd81796639046b992db03e7a6749771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:06:57.562564 19700 tablet_bootstrap.cc:654] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:57.564679 19700 tablet_bootstrap.cc:492] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5: No bootstrap required, opened a new log
I20260806 16:06:57.564886 19700 ts_tablet_manager.cc:1397] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.002s
I20260806 16:06:57.565075 19701 tablet_bootstrap.cc:492] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75: Bootstrap starting.
I20260806 16:06:57.565526 19700 raft_consensus.cc:348] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.565672 19700 raft_consensus.cc:374] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:57.565706 19700 raft_consensus.cc:729] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 455f7250bda648c590c6c999665d81d5, State: Initialized, Role: FOLLOWER
I20260806 16:06:57.565917 19700 consensus_queue.cc:260] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.566007 19702 tablet_bootstrap.cc:492] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9: No bootstrap required, opened a new log
I20260806 16:06:57.566128 19702 ts_tablet_manager.cc:1397] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 16:06:57.566144 19701 tablet_bootstrap.cc:654] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75: Neither blocks nor log segments found. Creating new log.
I20260806 16:06:57.566650 19702 raft_consensus.cc:348] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.566792 19702 raft_consensus.cc:374] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:57.566852 19702 raft_consensus.cc:729] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5f2f647cde43dc908cacc2885682a9, State: Initialized, Role: FOLLOWER
I20260806 16:06:57.566993 19700 ts_tablet_manager.cc:1428] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:57.567093 19702 consensus_queue.cc:260] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.568377 19702 ts_tablet_manager.cc:1428] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 16:06:57.571094 19701 tablet_bootstrap.cc:492] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75: No bootstrap required, opened a new log
I20260806 16:06:57.571257 19701 ts_tablet_manager.cc:1397] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 16:06:57.571818 19701 raft_consensus.cc:348] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.571985 19701 raft_consensus.cc:374] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:06:57.572027 19701 raft_consensus.cc:729] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ed5b21cd654ed281efae740a991a75, State: Initialized, Role: FOLLOWER
I20260806 16:06:57.572472 19701 consensus_queue.cc:260] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.572835 19701 ts_tablet_manager.cc:1428] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 16:06:57.862663 19714 raft_consensus.cc:482] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:06:57.863121 19714 raft_consensus.cc:504] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.863844 19714 leader_election.cc:288] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511), 83ed5b21cd654ed281efae740a991a75 (127.8.49.3:35131)
I20260806 16:06:57.864789 19598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd81796639046b992db03e7a6749771" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ed5b21cd654ed281efae740a991a75" is_pre_election: true
I20260806 16:06:57.864789 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd81796639046b992db03e7a6749771" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455f7250bda648c590c6c999665d81d5" is_pre_election: true
I20260806 16:06:57.865039 19394 raft_consensus.cc:2454] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 0.
I20260806 16:06:57.865034 19598 raft_consensus.cc:2454] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 0.
I20260806 16:06:57.865612 19094 leader_election.cc:302] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 455f7250bda648c590c6c999665d81d5, 83ed5b21cd654ed281efae740a991a75, eb5f2f647cde43dc908cacc2885682a9; no voters:
I20260806 16:06:57.865962 19714 raft_consensus.cc:2781] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:06:57.866053 19714 raft_consensus.cc:482] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:06:57.866083 19714 raft_consensus.cc:3037] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:57.867542 19714 raft_consensus.cc:504] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.868021 19714 leader_election.cc:288] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 election: Requested vote from peers 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511), 83ed5b21cd654ed281efae740a991a75 (127.8.49.3:35131)
I20260806 16:06:57.868880 19598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd81796639046b992db03e7a6749771" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83ed5b21cd654ed281efae740a991a75"
I20260806 16:06:57.868880 19394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd81796639046b992db03e7a6749771" candidate_uuid: "eb5f2f647cde43dc908cacc2885682a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "455f7250bda648c590c6c999665d81d5"
I20260806 16:06:57.869095 19394 raft_consensus.cc:3037] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:57.869095 19598 raft_consensus.cc:3037] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:06:57.870512 19394 raft_consensus.cc:2454] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 1.
I20260806 16:06:57.870512 19598 raft_consensus.cc:2454] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb5f2f647cde43dc908cacc2885682a9 in term 1.
I20260806 16:06:57.870978 19094 leader_election.cc:302] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ed5b21cd654ed281efae740a991a75, eb5f2f647cde43dc908cacc2885682a9; no voters:
I20260806 16:06:57.871256 19714 raft_consensus.cc:2781] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:06:57.871531 19714 raft_consensus.cc:686] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 1 LEADER]: Becoming Leader. State: Replica: eb5f2f647cde43dc908cacc2885682a9, State: Running, Role: LEADER
I20260806 16:06:57.871757 19714 consensus_queue.cc:237] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } }
I20260806 16:06:57.874156 18955 catalog_manager.cc:5495] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "eb5f2f647cde43dc908cacc2885682a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5f2f647cde43dc908cacc2885682a9" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 } health_report { overall_health: UNKNOWN } } }
I20260806 16:06:57.964032 19836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:57.960205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58778 (local address 127.8.49.1:41253)
0806 16:06:57.960559 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 16:06:57.960563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:06:57.960576 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:06:57.960626 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:06:57.960628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:06:57.960678 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:06:57.960860 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:06:57.961153 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.961819 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.962771 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:06:57.963051 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:06:57.963172 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:06:57.963472 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:06:57.963564 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:06:57.963651 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 16:06:57.963755 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 16:06:57.963820 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":112,"threads_started":1}
I20260806 16:06:57.974552 19598 raft_consensus.cc:1261] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Refusing update from remote peer eb5f2f647cde43dc908cacc2885682a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:57.975086 19394 raft_consensus.cc:1261] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Refusing update from remote peer eb5f2f647cde43dc908cacc2885682a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:06:57.975252 19832 consensus_queue.cc:1035] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83ed5b21cd654ed281efae740a991a75" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 35131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:57.975684 19832 consensus_queue.cc:1035] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "455f7250bda648c590c6c999665d81d5" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:06:58.020283 18955 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:58.020548 18955 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:06:58.023209 18955 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 849154fda66c4e3ba2e32a77a50706e6: Sending DeleteTablet for 3 replicas of tablet ffd81796639046b992db03e7a6749771
I20260806 16:06:58.024207 19168 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd81796639046b992db03e7a6749771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:58 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:58760
I20260806 16:06:58.024384 19570 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd81796639046b992db03e7a6749771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:58 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:59790
I20260806 16:06:58.024528 19367 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd81796639046b992db03e7a6749771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:06:58 UTC) from {username='oryx', principal='oryx/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:52922
I20260806 16:06:58.024667 19801 tablet_replica.cc:330] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75: stopping tablet replica
I20260806 16:06:58.024833 19800 tablet_replica.cc:330] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9: stopping tablet replica
I20260806 16:06:58.024840 19801 raft_consensus.cc:2229] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:58.024966 19801 raft_consensus.cc:2258] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:58.025007 19800 raft_consensus.cc:2229] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:06:58.025282 19800 raft_consensus.cc:2258] T ffd81796639046b992db03e7a6749771 P eb5f2f647cde43dc908cacc2885682a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:58.025406 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 19068
I20260806 16:06:58.025501 19802 tablet_replica.cc:330] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5: stopping tablet replica
I20260806 16:06:58.025671 19802 raft_consensus.cc:2229] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:06:58.025812 19802 raft_consensus.cc:2258] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:06:58.036919 19801 ts_tablet_manager.cc:1905] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:58.037194 19802 ts_tablet_manager.cc:1905] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:06:58.040189 19801 ts_tablet_manager.cc:1918] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:58.040292 19801 log.cc:1196] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/ffd81796639046b992db03e7a6749771
I20260806 16:06:58.040549 19801 ts_tablet_manager.cc:1939] T ffd81796639046b992db03e7a6749771 P 83ed5b21cd654ed281efae740a991a75: Deleting consensus metadata
I20260806 16:06:58.042047 18943 catalog_manager.cc:4841] TS 83ed5b21cd654ed281efae740a991a75 (127.8.49.3:35131): tablet ffd81796639046b992db03e7a6749771 (table test-table [id=6330439223154eea9c3976804870acab]) successfully deleted
I20260806 16:06:58.042377 19802 ts_tablet_manager.cc:1918] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 16:06:58.042505 19802 log.cc:1196] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/ffd81796639046b992db03e7a6749771
I20260806 16:06:58.042735 19802 ts_tablet_manager.cc:1939] T ffd81796639046b992db03e7a6749771 P 455f7250bda648c590c6c999665d81d5: Deleting consensus metadata
I20260806 16:06:58.043774 18942 catalog_manager.cc:4841] TS 455f7250bda648c590c6c999665d81d5 (127.8.49.2:37511): tablet ffd81796639046b992db03e7a6749771 (table test-table [id=6330439223154eea9c3976804870acab]) successfully deleted
W20260806 16:06:58.051437 18940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:41253 (error 108)
I20260806 16:06:58.051749 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 19274
I20260806 16:06:58.059070 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:06:58.058387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41253 (local address 127.0.0.1:58794)
0806 16:06:58.058748 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 16:06:58.058923 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:41253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":125,"threads_started":1}
W20260806 16:06:58.059283 18940 catalog_manager.cc:4584] TS eb5f2f647cde43dc908cacc2885682a9 (127.8.49.1:41253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ffd81796639046b992db03e7a6749771: Network error: Client connection negotiation failed: client connection to 127.8.49.1:41253: connect: Connection refused (error 111)
I20260806 16:06:58.066250 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 19475
I20260806 16:06:58.085886 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 18913
2026-08-06T16:06:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (4590 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:58 dist-test-slave-7z32 krb5kdc[19849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:58 dist-test-slave-7z32 krb5kdc[19849](info): set up 2 sockets
Aug 06 16:06:58 dist-test-slave-7z32 krb5kdc[19849](info): commencing operation
krb5kdc: starting...
W20260806 16:06:59.473989 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.353s user 0.002s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:06:59 dist-test-slave-7z32 krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:06:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:06:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:06:59.656908 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:46691
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:34011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:46691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:06:59.798512 19869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:06:59.799031 19869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:06:59.799098 19869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:06:59.805540 19869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:06:59.805670 19869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:06:59.805693 19869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:06:59.805713 19869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:06:59.805729 19869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:06:59.812738 19869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:34011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:46691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:46691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:06:59.814461 19869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:06:59.815917 19869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:06:59.828311 19883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:06:59.828315 19880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:59.828697 19869 server_base.cc:1034] running on GCE node
W20260806 16:06:59.828349 19881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:06:59.829946 19869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:06:59.831704 19869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:06:59.835069 19869 hybrid_clock.cc:648] HybridClock initialized: now 1786032419835012 us; error 68 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T16:06:59Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1752 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:06:59 dist-test-slave-7z32 krb5kdc[19892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:06:59 dist-test-slave-7z32 krb5kdc[19892](info): set up 2 sockets
Aug 06 16:06:59 dist-test-slave-7z32 krb5kdc[19892](info): commencing operation
krb5kdc: starting...
W20260806 16:07:01.204717 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.330s user 0.006s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:07:01 dist-test-slave-7z32 krb5kdc[19892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:07:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:07:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:07:01.398574 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:45221
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:45221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:07:01.525700 19911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:01.526103 19911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:01.526183 19911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:01.530543 19911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:07:01.530682 19911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:01.530719 19911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:01.530761 19911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:07:01.530814 19911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:07:01.536772 19911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:45221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:45221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:01.538002 19911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:01.539167 19911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:01.550346 19918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:01.550670 19911 server_base.cc:1034] running on GCE node
W20260806 16:07:01.550339 19917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:01.550560 19920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:01.551802 19911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:01.553539 19911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:01.554795 19911 hybrid_clock.cc:648] HybridClock initialized: now 1786032421554740 us; error 63 us; skew 500 ppm
Aug 06 16:07:01 dist-test-slave-7z32 krb5kdc[19892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032421, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:01.559504 19911 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:07:01.561095 19911 webserver.cc:460] Webserver started at http://127.8.49.62:39745/ using document root <none> and password file <none>
I20260806 16:07:01.562003 19911 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:01.562134 19911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:01.562462 19911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:01.565781 19911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "b24a6dac2a6e491495d4878a2afd70eb"
format_stamp: "Formatted at 2026-08-06 16:07:01 on dist-test-slave-7z32"
server_key: "6b47a7d24c3854f6060d44d22f9f3501"
server_key_iv: "6dcc38f4c5452ea7bf0980058694a399"
server_key_version: "encryptionkey@0"
I20260806 16:07:01.566726 19911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "b24a6dac2a6e491495d4878a2afd70eb"
format_stamp: "Formatted at 2026-08-06 16:07:01 on dist-test-slave-7z32"
server_key: "6b47a7d24c3854f6060d44d22f9f3501"
server_key_iv: "6dcc38f4c5452ea7bf0980058694a399"
server_key_version: "encryptionkey@0"
I20260806 16:07:01.572817 19911 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260806 16:07:01.577499 19928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:01.579869 19911 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:07:01.580166 19911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "b24a6dac2a6e491495d4878a2afd70eb"
format_stamp: "Formatted at 2026-08-06 16:07:01 on dist-test-slave-7z32"
server_key: "6b47a7d24c3854f6060d44d22f9f3501"
server_key_iv: "6dcc38f4c5452ea7bf0980058694a399"
server_key_version: "encryptionkey@0"
I20260806 16:07:01.580425 19911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:01.615455 19911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:01.620780 19911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:01.621361 19911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:01.634012 19911 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:45221
I20260806 16:07:01.634040 20005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:45221 every 8 connection(s)
I20260806 16:07:01.635533 19911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:07:01.639962 20006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:07:01.642455 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 19911
I20260806 16:07:01.642633 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:07:01.643060 8388 external_mini_cluster.cc:1428] Setting key 416d8df866127edc2c276ef805b51f2b
I20260806 16:07:01.651254 20006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb: Bootstrap starting.
I20260806 16:07:01.656159 20006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb: Neither blocks nor log segments found. Creating new log.
I20260806 16:07:01.657722 20006 log.cc:824] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:07:01 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:07:01.661275 20006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb: No bootstrap required, opened a new log
I20260806 16:07:01.665128 20006 raft_consensus.cc:348] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } }
I20260806 16:07:01.665489 20006 raft_consensus.cc:374] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:07:01.665586 20006 raft_consensus.cc:729] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24a6dac2a6e491495d4878a2afd70eb, State: Initialized, Role: FOLLOWER
I20260806 16:07:01.666204 20006 consensus_queue.cc:260] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } }
I20260806 16:07:01.666420 20006 raft_consensus.cc:388] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:07:01.666527 20006 raft_consensus.cc:482] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:07:01.666646 20006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:07:01.668008 20006 raft_consensus.cc:504] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } }
I20260806 16:07:01.668457 20006 leader_election.cc:302] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b24a6dac2a6e491495d4878a2afd70eb; no voters:
I20260806 16:07:01.668921 20006 leader_election.cc:288] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:07:01.669024 20012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:07:01.669243 20012 raft_consensus.cc:686] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 1 LEADER]: Becoming Leader. State: Replica: b24a6dac2a6e491495d4878a2afd70eb, State: Running, Role: LEADER
I20260806 16:07:01.669579 20012 consensus_queue.cc:237] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } }
I20260806 16:07:01.670512 20006 sys_catalog.cc:564] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:07:01.670989 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:01.646055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54532 (local address 127.8.49.62:45221)
0806 16:07:01.647403 (+ 1348us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:01.647435 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:01.647505 (+ 70us) server_negotiation.cc:407] Connection header received
0806 16:07:01.648618 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:01.648651 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:01.649456 (+ 805us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:01.650040 (+ 584us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:01.651770 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:01.653114 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:01.654965 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:01.656262 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:01.656586 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:01.661193 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:01.661201 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:01.661229 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:07:01.661273 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:01.664616 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:01.665379 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:01.665387 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:01.665395 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:01.665611 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:01.666196 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:01.666200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:01.666201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:01.667299 (+ 1098us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:01.667711 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:01.668258 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 16:07:01.668871 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":918,"thread_start_us":513,"threads_started":1}
I20260806 16:07:01.671509 20014 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b24a6dac2a6e491495d4878a2afd70eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } } }
I20260806 16:07:01.671686 20014 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [sys.catalog]: This master's current role is: LEADER
I20260806 16:07:01.672092 20017 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b24a6dac2a6e491495d4878a2afd70eb. Latest consensus state: current_term: 1 leader_uuid: "b24a6dac2a6e491495d4878a2afd70eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } } }
I20260806 16:07:01.672223 20017 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [sys.catalog]: This master's current role is: LEADER
I20260806 16:07:01.672358 20024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:07:01.677364 20024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:07:01.687531 20024 catalog_manager.cc:1347] Generated new cluster ID: 1d59c9b6b88845e2be5daeef5c92fb07
I20260806 16:07:01.687683 20024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:07:01.705581 20024 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:07:01.707536 20024 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:07:01.723412 20024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb: Generated new TSK 0
I20260806 16:07:01.724663 20024 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:07:01.772754 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45221
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:07:01.936848 20045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:01.937302 20045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:01.937364 20045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:01.942934 20045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:01.943073 20045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:01.943150 20045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:07:01.950167 20045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:01.951913 20045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:01.953416 20045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:01.967974 20054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:01.968086 20058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:01.967974 20051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:01.968688 20045 server_base.cc:1034] running on GCE node
I20260806 16:07:01.969350 20045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:01.970329 20045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:01.971763 20045 hybrid_clock.cc:648] HybridClock initialized: now 1786032421971710 us; error 106 us; skew 500 ppm
Aug 06 16:07:01 dist-test-slave-7z32 krb5kdc[19892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032421, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:01.975921 20045 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:07:01.977205 20045 webserver.cc:460] Webserver started at http://127.8.49.1:43845/ using document root <none> and password file <none>
I20260806 16:07:01.977914 20045 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:01.978029 20045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:01.978344 20045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:01.980892 20045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "5867e360a68640409e639041f7c3bfa2"
format_stamp: "Formatted at 2026-08-06 16:07:01 on dist-test-slave-7z32"
server_key: "2e189b3ac1951ed38cc71b61b2a37170"
server_key_iv: "d95b5098dc0db64c4974eeada61d2c1c"
server_key_version: "encryptionkey@0"
I20260806 16:07:01.982152 20045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "5867e360a68640409e639041f7c3bfa2"
format_stamp: "Formatted at 2026-08-06 16:07:01 on dist-test-slave-7z32"
server_key: "2e189b3ac1951ed38cc71b61b2a37170"
server_key_iv: "d95b5098dc0db64c4974eeada61d2c1c"
server_key_version: "encryptionkey@0"
I20260806 16:07:01.988631 20045 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260806 16:07:01.992702 20067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:01.994647 20045 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:07:01.994997 20045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "5867e360a68640409e639041f7c3bfa2"
format_stamp: "Formatted at 2026-08-06 16:07:01 on dist-test-slave-7z32"
server_key: "2e189b3ac1951ed38cc71b61b2a37170"
server_key_iv: "d95b5098dc0db64c4974eeada61d2c1c"
server_key_version: "encryptionkey@0"
I20260806 16:07:01.995203 20045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:02.016691 20045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:02.020980 20045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:02.021190 20045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:02.021903 20045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:02.023106 20045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:02.023168 20045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:02.023234 20045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:02.023252 20045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:02.036572 20045 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:40847
I20260806 16:07:02.036688 20232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:40847 every 8 connection(s)
I20260806 16:07:02.037777 20045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:07:02.046450 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 20045
I20260806 16:07:02.046608 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:07:02.047009 8388 external_mini_cluster.cc:1428] Setting key 0432b110ebbf34f9a6ed314b98895b5a
Aug 06 16:07:02 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032421, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:07:02.060357 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:02.040684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:56195 (local address 127.8.49.62:45221)
0806 16:07:02.040951 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:02.040956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:02.041985 (+ 1029us) server_negotiation.cc:407] Connection header received
0806 16:07:02.044106 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:02.044110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:02.044183 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:02.044287 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:02.045926 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.046740 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.048676 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.049052 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.049198 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:02.054325 (+ 5127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:02.054339 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:02.054341 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:07:02.054411 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:02.056590 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:02.057575 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:02.057580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:02.057582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:02.057706 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:02.058225 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:02.058230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:02.058235 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:02.058597 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:02.058744 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:02.059865 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0806 16:07:02.060089 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260806 16:07:02.061909 20238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:02.041114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:56195)
0806 16:07:02.041779 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:02.041832 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:02.043735 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:02.044611 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:02.044622 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:02.045145 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:02.045734 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:02.045752 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.047008 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.047013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:02.048476 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:02.048495 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.049338 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.049346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:02.049709 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:02.050648 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:02.050682 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:02.053939 (+ 3257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:02.056881 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:02.056892 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:02.056930 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:02.057412 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:02.057876 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:02.057879 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:02.057881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:02.058086 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:02.058810 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:02.058818 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:02.059367 (+ 549us) client_negotiation.cc:769] Sending connection context
0806 16:07:02.059628 (+ 261us) client_negotiation.cc:240] Negotiation successful
0806 16:07:02.060305 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":154,"threads_started":1}
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
I20260806 16:07:02.064092 20234 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45221
I20260806 16:07:02.064527 20234 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:02.065279 20234 heartbeater.cc:511] Master 127.8.49.62:45221 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:07:02.067801 19955 ts_manager.cc:194] Registered new tserver with Master: 5867e360a68640409e639041f7c3bfa2 (127.8.49.1:40847)
I20260806 16:07:02.070083 19955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:56195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:07:02.095600 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45221
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:07:02.244719 20242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:02.245088 20242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:02.245141 20242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:02.249775 20242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:02.249874 20242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:02.249939 20242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:07:02.255936 20242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:02.257172 20242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:02.258278 20242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:02.270879 20247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:02.270910 20248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:02.270910 20251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:02.271945 20242 server_base.cc:1034] running on GCE node
I20260806 16:07:02.272961 20242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:02.274088 20242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:02.275357 20242 hybrid_clock.cc:648] HybridClock initialized: now 1786032422275328 us; error 57 us; skew 500 ppm
Aug 06 16:07:02 dist-test-slave-7z32 krb5kdc[19892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:02.279353 20242 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:07:02.280761 20242 webserver.cc:460] Webserver started at http://127.8.49.2:45807/ using document root <none> and password file <none>
I20260806 16:07:02.281565 20242 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:02.281649 20242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:02.281947 20242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:02.284832 20242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "f9f468e977cb4717927e65e4a297b369"
format_stamp: "Formatted at 2026-08-06 16:07:02 on dist-test-slave-7z32"
server_key: "0d128ccaaa7651921a98532205aa74f7"
server_key_iv: "b314429692ba79cf9f51be482b4571b5"
server_key_version: "encryptionkey@0"
I20260806 16:07:02.285588 20242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "f9f468e977cb4717927e65e4a297b369"
format_stamp: "Formatted at 2026-08-06 16:07:02 on dist-test-slave-7z32"
server_key: "0d128ccaaa7651921a98532205aa74f7"
server_key_iv: "b314429692ba79cf9f51be482b4571b5"
server_key_version: "encryptionkey@0"
I20260806 16:07:02.291695 20242 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:07:02.296204 20261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:02.298435 20242 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:07:02.298694 20242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "f9f468e977cb4717927e65e4a297b369"
format_stamp: "Formatted at 2026-08-06 16:07:02 on dist-test-slave-7z32"
server_key: "0d128ccaaa7651921a98532205aa74f7"
server_key_iv: "b314429692ba79cf9f51be482b4571b5"
server_key_version: "encryptionkey@0"
I20260806 16:07:02.298990 20242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:02.346230 20242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:02.351226 20242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:02.351548 20242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:02.352521 20242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:02.353783 20242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:02.353888 20242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:02.353976 20242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:02.354025 20242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:02.368697 20242 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:45349
I20260806 16:07:02.368763 20423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:45349 every 8 connection(s)
I20260806 16:07:02.370369 20242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:07:02.378872 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 20242
I20260806 16:07:02.379009 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:07:02.379323 8388 external_mini_cluster.cc:1428] Setting key 2738a6e0805c7bb830b279082f805edd
Aug 06 16:07:02 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:07:02.392856 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:02.373460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:53739 (local address 127.8.49.62:45221)
0806 16:07:02.373693 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:02.373700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:02.375492 (+ 1792us) server_negotiation.cc:407] Connection header received
0806 16:07:02.376632 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:02.376637 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:02.376760 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:02.376896 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:02.378741 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.379697 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.381724 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.382125 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.382271 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:02.387048 (+ 4777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:02.387057 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:02.387064 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:07:02.387128 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:02.389787 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:02.390787 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:02.390799 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:02.390800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:02.390984 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:02.391425 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:02.391429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:02.391431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:02.391672 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:02.391790 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:02.392483 (+ 693us) server_negotiation.cc:299] Negotiation successful
0806 16:07:02.392669 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 16:07:02.393952 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:02.374194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:53739)
0806 16:07:02.375270 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:02.375330 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:02.376283 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:02.377177 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:02.377186 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:02.377729 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:02.378314 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:02.378386 (+ 72us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.379969 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.379982 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:02.381339 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:02.381354 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.382411 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.382418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:02.382721 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:02.383739 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:02.383769 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:02.386591 (+ 2822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:02.390094 (+ 3503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:02.390103 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:02.390125 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:02.390603 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:02.391119 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:02.391122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:02.391124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:02.391306 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:02.391819 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:02.391828 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:02.392195 (+ 367us) client_negotiation.cc:769] Sending connection context
0806 16:07:02.392430 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 16:07:02.392843 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":211,"threads_started":1}
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:07:02.396744 20424 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45221
I20260806 16:07:02.397231 20424 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:02.397955 20424 heartbeater.cc:511] Master 127.8.49.62:45221 requested a full tablet report, sending...
I20260806 16:07:02.399978 19955 ts_manager.cc:194] Registered new tserver with Master: f9f468e977cb4717927e65e4a297b369 (127.8.49.2:45349)
I20260806 16:07:02.400707 19955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:53739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:07:02.431525 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45221
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:07:02.591773 20430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:02.592258 20430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:02.592360 20430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:02.597764 20430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:02.597965 20430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:02.598315 20430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:07:02.608130 20430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:02.609946 20430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:02.611750 20430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:02.627370 20438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:02.627447 20442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:02.627408 20437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:02.628099 20430 server_base.cc:1034] running on GCE node
I20260806 16:07:02.628854 20430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:02.630113 20430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:02.631417 20430 hybrid_clock.cc:648] HybridClock initialized: now 1786032422631399 us; error 78 us; skew 500 ppm
Aug 06 16:07:02 dist-test-slave-7z32 krb5kdc[19892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:02.636477 20430 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:07:02.638163 20430 webserver.cc:460] Webserver started at http://127.8.49.3:42289/ using document root <none> and password file <none>
I20260806 16:07:02.639158 20430 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:02.639288 20430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:02.639624 20430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:02.642781 20430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "9f2638c79c30499bb0489bb462807851"
format_stamp: "Formatted at 2026-08-06 16:07:02 on dist-test-slave-7z32"
server_key: "3389b400ed4ad719a57eb308684a2280"
server_key_iv: "05b989296f6345ffbc6928988ec47d7a"
server_key_version: "encryptionkey@0"
I20260806 16:07:02.643730 20430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "9f2638c79c30499bb0489bb462807851"
format_stamp: "Formatted at 2026-08-06 16:07:02 on dist-test-slave-7z32"
server_key: "3389b400ed4ad719a57eb308684a2280"
server_key_iv: "05b989296f6345ffbc6928988ec47d7a"
server_key_version: "encryptionkey@0"
I20260806 16:07:02.649501 20430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 16:07:02.654206 20449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:02.656715 20430 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:07:02.657001 20430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "9f2638c79c30499bb0489bb462807851"
format_stamp: "Formatted at 2026-08-06 16:07:02 on dist-test-slave-7z32"
server_key: "3389b400ed4ad719a57eb308684a2280"
server_key_iv: "05b989296f6345ffbc6928988ec47d7a"
server_key_version: "encryptionkey@0"
I20260806 16:07:02.657203 20430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:02.683753 20430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:02.690088 20430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:02.690449 20430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:02.691493 20430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:02.692751 20430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:02.692850 20430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:02.692953 20430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:02.693006 20430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:02.710559 20430 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:42381
I20260806 16:07:02.710677 20600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:42381 every 8 connection(s)
I20260806 16:07:02.712182 20430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:07:02.714344 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 20430
I20260806 16:07:02.714530 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:07:02.715076 8388 external_mini_cluster.cc:1428] Setting key 19a39e2ac760fd338f549922426008aa
Aug 06 16:07:02 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:07:02.735706 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:02.715312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:52223 (local address 127.8.49.62:45221)
0806 16:07:02.715808 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:02.715816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:02.717047 (+ 1231us) server_negotiation.cc:407] Connection header received
0806 16:07:02.719578 (+ 2531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:02.719582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:02.719650 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:02.719749 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:02.722195 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.722883 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.724877 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.725180 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.725308 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:02.729613 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:02.729617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:02.729620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:07:02.729698 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:02.732255 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:02.733358 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:02.733383 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:02.733385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:02.733535 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:02.734101 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:02.734103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:02.734105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:02.734369 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:02.734466 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:02.735391 (+ 925us) server_negotiation.cc:299] Negotiation successful
0806 16:07:02.735559 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 16:07:02.736696 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:02.715859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:52223)
0806 16:07:02.716804 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:02.716911 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:02.719092 (+ 2181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:02.720102 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:02.720113 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:02.721077 (+ 964us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:02.721911 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:02.721935 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.723082 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.723086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:02.724609 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:02.724629 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:02.725463 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:02.725467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:02.725702 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:02.726675 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:02.726699 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:02.729246 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:02.732513 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:02.732527 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:02.732558 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:02.733186 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:02.733691 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:02.733697 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:02.733701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:02.733971 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:02.734495 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:02.734502 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:02.734968 (+ 466us) client_negotiation.cc:769] Sending connection context
0806 16:07:02.735226 (+ 258us) client_negotiation.cc:240] Negotiation successful
0806 16:07:02.735584 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":213,"threads_started":1}
I20260806 16:07:02.738615 20602 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45221
I20260806 16:07:02.739324 20602 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:02.740334 20602 heartbeater.cc:511] Master 127.8.49.62:45221 requested a full tablet report, sending...
I20260806 16:07:02.742151 19955 ts_manager.cc:194] Registered new tserver with Master: 9f2638c79c30499bb0489bb462807851 (127.8.49.3:42381)
I20260806 16:07:02.743763 19955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:52223
I20260806 16:07:02.746069 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab.
Entry for principal oryx/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab.
WARNING: no policy specified for oryx/127.8.49.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab.
Entry for principal oryx/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab.
WARNING: no policy specified for oryx/127.8.49.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.49.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab.
Entry for principal oryx/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab.
I20260806 16:07:02.867885 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 20045
I20260806 16:07:02.885358 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 20242
I20260806 16:07:02.900072 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 20430
I20260806 16:07:02.934743 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 19911
I20260806 16:07:02.949056 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:45221
--webserver_interface=127.8.49.62
--webserver_port=39745
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:45221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:07:03.112931 20617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:03.113521 20617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:03.113631 20617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:03.119591 20617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:07:03.119750 20617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:03.119791 20617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:03.119839 20617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:07:03.119901 20617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:07:03.127985 20617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:45221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:45221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=39745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:03.129810 20617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:03.131836 20617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:03.147277 20625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:03.147276 20624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:03.147337 20628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:03.148025 20617 server_base.cc:1034] running on GCE node
I20260806 16:07:03.149020 20617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:03.151736 20617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:03.155432 20617 hybrid_clock.cc:648] HybridClock initialized: now 1786032423155355 us; error 79 us; skew 500 ppm
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:03.161752 20617 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:07:03.163992 20617 webserver.cc:460] Webserver started at http://127.8.49.62:39745/ using document root <none> and password file <none>
I20260806 16:07:03.165225 20617 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:03.165439 20617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:03.175472 20617 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 16:07:03.179910 20639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:03.183120 20617 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:07:03.183645 20617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "b24a6dac2a6e491495d4878a2afd70eb"
format_stamp: "Formatted at 2026-08-06 16:07:01 on dist-test-slave-7z32"
server_key: "6b47a7d24c3854f6060d44d22f9f3501"
server_key_iv: "6dcc38f4c5452ea7bf0980058694a399"
server_key_version: "encryptionkey@0"
I20260806 16:07:03.185046 20617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:03.214726 20617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:03.220341 20617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:03.220597 20617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:03.232139 20617 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:45221
I20260806 16:07:03.232290 20715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:45221 every 8 connection(s)
I20260806 16:07:03.234077 20617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:07:03.240790 20716 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:07:03.243099 20716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb: Bootstrap starting.
I20260806 16:07:03.243693 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 20617
I20260806 16:07:03.244328 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:40847
--local_ip_for_outbound_sockets=127.8.49.1
--tserver_master_addrs=127.8.49.62:45221
--webserver_port=43845
--webserver_interface=127.8.49.1
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.49.1
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:07:03.264006 20716 log.cc:824] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb: Log is configured to *not* fsync() on all Append() calls
I20260806 16:07:03.274233 20716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:07:03.274740 20716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb: Bootstrap complete.
I20260806 16:07:03.279942 20716 raft_consensus.cc:348] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } }
I20260806 16:07:03.280473 20716 raft_consensus.cc:729] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24a6dac2a6e491495d4878a2afd70eb, State: Initialized, Role: FOLLOWER
I20260806 16:07:03.281093 20716 consensus_queue.cc:260] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } }
I20260806 16:07:03.281234 20716 raft_consensus.cc:388] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:07:03.281348 20716 raft_consensus.cc:482] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:07:03.281487 20716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:07:03.284241 20716 raft_consensus.cc:504] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } }
I20260806 16:07:03.284884 20716 leader_election.cc:302] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b24a6dac2a6e491495d4878a2afd70eb; no voters:
I20260806 16:07:03.285429 20716 leader_election.cc:288] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 16:07:03.285554 20720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:07:03.285776 20720 raft_consensus.cc:686] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [term 2 LEADER]: Becoming Leader. State: Replica: b24a6dac2a6e491495d4878a2afd70eb, State: Running, Role: LEADER
I20260806 16:07:03.286124 20720 consensus_queue.cc:237] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } }
I20260806 16:07:03.287015 20716 sys_catalog.cc:564] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:07:03.287299 20723 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b24a6dac2a6e491495d4878a2afd70eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } } }
I20260806 16:07:03.287313 20720 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b24a6dac2a6e491495d4878a2afd70eb. Latest consensus state: current_term: 2 leader_uuid: "b24a6dac2a6e491495d4878a2afd70eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a6dac2a6e491495d4878a2afd70eb" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45221 } } }
I20260806 16:07:03.287777 20723 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [sys.catalog]: This master's current role is: LEADER
I20260806 16:07:03.287807 20720 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb [sys.catalog]: This master's current role is: LEADER
I20260806 16:07:03.288640 20725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:07:03.293175 20725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:07:03.294174 20725 catalog_manager.cc:1259] Loaded cluster ID: 1d59c9b6b88845e2be5daeef5c92fb07
I20260806 16:07:03.294291 20725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:07:03.296764 20725 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:07:03.297658 20725 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b24a6dac2a6e491495d4878a2afd70eb: Loaded TSK: 0
I20260806 16:07:03.299149 20725 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:07:03.422098 20718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:03.422523 20718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:03.422617 20718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:03.427726 20718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:03.427871 20718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:03.427981 20718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:07:03.436074 20718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.1.keytab
--principal=oryx/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:40847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=43845
--tserver_master_addrs=127.8.49.62:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:03.437480 20718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:03.438807 20718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:03.450685 20749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:03.450752 20753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:03.450675 20751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:03.451279 20718 server_base.cc:1034] running on GCE node
I20260806 16:07:03.451809 20718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:03.452585 20718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:03.454010 20718 hybrid_clock.cc:648] HybridClock initialized: now 1786032423453975 us; error 67 us; skew 500 ppm
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032423, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:03.457412 20718 init.cc:377] Logged in from keytab as oryx/127.8.49.1@KRBTEST.COM (short username oryx)
I20260806 16:07:03.458598 20718 webserver.cc:460] Webserver started at http://127.8.49.1:43845/ using document root <none> and password file <none>
I20260806 16:07:03.459352 20718 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:03.459443 20718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:03.464707 20718 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:07:03.468106 20762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:03.470156 20718 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:07:03.470547 20718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "5867e360a68640409e639041f7c3bfa2"
format_stamp: "Formatted at 2026-08-06 16:07:01 on dist-test-slave-7z32"
server_key: "2e189b3ac1951ed38cc71b61b2a37170"
server_key_iv: "d95b5098dc0db64c4974eeada61d2c1c"
server_key_version: "encryptionkey@0"
I20260806 16:07:03.471372 20718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:03.489811 20718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:03.493727 20718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:03.493992 20718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:03.494805 20718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:03.496040 20718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:03.496122 20718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:03.496201 20718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:03.496253 20718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:03.508738 20718 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:40847
I20260806 16:07:03.508782 20925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:40847 every 8 connection(s)
I20260806 16:07:03.509968 20718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:07:03.517539 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 20718
I20260806 16:07:03.518301 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:45349
--local_ip_for_outbound_sockets=127.8.49.2
--tserver_master_addrs=127.8.49.62:45221
--webserver_port=45807
--webserver_interface=127.8.49.2
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.49.2
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:03.528760 20928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:03.513131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:37881)
0806 16:07:03.513850 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:03.513924 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:03.515557 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:03.517342 (+ 1785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:03.517354 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:03.518038 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:03.519075 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.519112 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.520899 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.520908 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.522319 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.522335 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.523905 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.523911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.524211 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.525037 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:03.525057 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:03.527839 (+ 2782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":134,"threads_started":1}
W20260806 16:07:03.529484 20926 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:45221 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.49.62:45221: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
W20260806 16:07:03.530337 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:03.513152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:37881 (local address 127.8.49.62:45221)
0806 16:07:03.513894 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:03.513918 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:03.514096 (+ 178us) server_negotiation.cc:407] Connection header received
0806 16:07:03.516193 (+ 2097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:03.516208 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:03.516631 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:03.516990 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:03.519398 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.520622 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.522561 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.523579 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.523858 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.528460 (+ 4602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:37881: BlockingRecv error: recv got EOF from 127.8.49.1:37881 (error 108)
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":238,"threads_started":1}
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:03.538621 20928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:03.530207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:34037)
0806 16:07:03.530420 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:03.530438 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:03.530541 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:03.531370 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:03.531373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:03.531561 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:03.531687 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.531693 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.533609 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.533612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.534290 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.534308 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.535218 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.535222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.535323 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.535951 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:03.535964 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:03.538413 (+ 2449us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120}
W20260806 16:07:03.539211 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:03.530312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:34037 (local address 127.8.49.62:45221)
0806 16:07:03.530968 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:03.530976 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:03.531006 (+ 30us) server_negotiation.cc:407] Connection header received
0806 16:07:03.531088 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:03.531093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:03.531184 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:03.531330 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:03.531854 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.533390 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.534479 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.535025 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.535205 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.539038 (+ 3833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:34037: BlockingRecv error: recv got EOF from 127.8.49.1:34037 (error 108)
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":186,"threads_started":1}
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:03.547343 20928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:03.539662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:37845)
0806 16:07:03.539824 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:03.539842 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:03.539962 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:03.540277 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:03.540280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:03.540489 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:03.540581 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.540587 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.541462 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.541468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.542590 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.542612 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.543394 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.543399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.543541 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.544241 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:03.544260 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:03.547123 (+ 2863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 16:07:03.547741 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:03.539741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:37845 (local address 127.8.49.62:45221)
0806 16:07:03.539866 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:03.539873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:03.539894 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:07:03.540090 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:03.540095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:03.540152 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:03.540254 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:03.540705 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.541290 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.542906 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.543206 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.543323 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.547378 (+ 4055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:37845: BlockingRecv error: recv got EOF from 127.8.49.1:37845 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 16:07:03.547899 20926 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:07:03.698674 20933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:03.699225 20933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:03.699324 20933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:03.705580 20933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:03.705775 20933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:03.705915 20933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:07:03.715240 20933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.2.keytab
--principal=oryx/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:45349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=45807
--tserver_master_addrs=127.8.49.62:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:03.717058 20933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:03.718658 20933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:03.734578 20944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:03.734683 20943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:03.734578 20947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:03.735164 20933 server_base.cc:1034] running on GCE node
I20260806 16:07:03.735982 20933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:03.737046 20933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:03.740504 20933 hybrid_clock.cc:648] HybridClock initialized: now 1786032423740426 us; error 79 us; skew 500 ppm
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032423, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:03.745455 20933 init.cc:377] Logged in from keytab as oryx/127.8.49.2@KRBTEST.COM (short username oryx)
I20260806 16:07:03.747205 20933 webserver.cc:460] Webserver started at http://127.8.49.2:45807/ using document root <none> and password file <none>
I20260806 16:07:03.748072 20933 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:03.748229 20933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:03.754752 20933 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 16:07:03.759343 20958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:03.761586 20933 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:07:03.761900 20933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "f9f468e977cb4717927e65e4a297b369"
format_stamp: "Formatted at 2026-08-06 16:07:02 on dist-test-slave-7z32"
server_key: "0d128ccaaa7651921a98532205aa74f7"
server_key_iv: "b314429692ba79cf9f51be482b4571b5"
server_key_version: "encryptionkey@0"
I20260806 16:07:03.762779 20933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:03.788763 20933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:03.793994 20933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:03.794281 20933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:03.795320 20933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:03.796612 20933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:03.796700 20933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:03.796794 20933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:03.796837 20933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:03.810346 20933 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:45349
I20260806 16:07:03.810472 21117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:45349 every 8 connection(s)
I20260806 16:07:03.812239 20933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:07:03.813254 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 20933
I20260806 16:07:03.813938 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:42381
--local_ip_for_outbound_sockets=127.8.49.3
--tserver_master_addrs=127.8.49.62:45221
--webserver_port=42289
--webserver_interface=127.8.49.3
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.49.3
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:07:03.831274 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:03.815624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:33443 (local address 127.8.49.62:45221)
0806 16:07:03.815833 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:03.815841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:03.817292 (+ 1451us) server_negotiation.cc:407] Connection header received
0806 16:07:03.819344 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:03.819352 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:03.819466 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:03.819636 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:03.821880 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.822974 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.824891 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.825398 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.825621 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.831106 (+ 5485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:33443: BlockingRecv error: recv got EOF from 127.8.49.2:33443 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 16:07:03.832504 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:03.816112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:33443)
0806 16:07:03.816927 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:03.817041 (+ 114us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:03.818972 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:03.820069 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:03.820080 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:03.820681 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:03.821491 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.821514 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.823297 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.823301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.824653 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.824669 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.825715 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.825719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.825975 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.827104 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:03.827129 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:03.830893 (+ 3764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":202,"threads_started":1}
W20260806 16:07:03.833459 21119 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:45221 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.49.62:45221: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:03.840955 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:03.834199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:36245)
0806 16:07:03.834385 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:03.834411 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:03.834558 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:03.834896 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:03.834899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:03.835134 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:03.835278 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.835284 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.836142 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.836146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.836758 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.836767 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.837251 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.837255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.837357 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.837996 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:03.838010 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:03.840802 (+ 2792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260806 16:07:03.841624 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:03.834381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:36245 (local address 127.8.49.62:45221)
0806 16:07:03.834559 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:03.834565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:03.834584 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:07:03.834691 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:03.834693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:03.834744 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:03.834876 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:03.835416 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.836019 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.836887 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.837122 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.837240 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.841091 (+ 3851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:36245: BlockingRecv error: recv got EOF from 127.8.49.2:36245 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:03 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:03.850193 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:03.841917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:44253)
0806 16:07:03.842148 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:03.842168 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:03.842296 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:03.842892 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:03.842898 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:03.843137 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:03.843292 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.843300 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.844252 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.844256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.845212 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:03.845228 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.845951 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.845957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:03.846062 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.846934 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:03.846951 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:03.850018 (+ 3067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94}
W20260806 16:07:03.850648 21119 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:07:03.851047 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:03.842235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:44253 (local address 127.8.49.62:45221)
0806 16:07:03.842411 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:03.842417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:03.842438 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:07:03.842508 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:03.842515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:03.842585 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:03.842799 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:03.843468 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.844104 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.845399 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:03.845694 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:03.845899 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:03.850886 (+ 4987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:44253: BlockingRecv error: recv got EOF from 127.8.49.2:44253 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 16:07:03.977017 21122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:03.977422 21122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:03.977492 21122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:03.983937 21122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:03.984066 21122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:03.984136 21122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:07:03.993250 21122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/krb5kdc/oryx_127.8.49.3.keytab
--principal=oryx/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:42381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=42289
--tserver_master_addrs=127.8.49.62:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:03.995189 21122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:03.996471 21122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:04.012967 21130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:04.012944 21133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:04.012950 21129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:04.013662 21122 server_base.cc:1034] running on GCE node
I20260806 16:07:04.014371 21122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:04.015513 21122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:04.017028 21122 hybrid_clock.cc:648] HybridClock initialized: now 1786032424016931 us; error 105 us; skew 500 ppm
Aug 06 16:07:04 dist-test-slave-7z32 krb5kdc[19892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032424, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:04.022071 21122 init.cc:377] Logged in from keytab as oryx/127.8.49.3@KRBTEST.COM (short username oryx)
I20260806 16:07:04.023581 21122 webserver.cc:460] Webserver started at http://127.8.49.3:42289/ using document root <none> and password file <none>
I20260806 16:07:04.024413 21122 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:04.024529 21122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:04.032050 21122 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:07:04.036573 21143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:04.039409 21122 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:07:04.039705 21122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "9f2638c79c30499bb0489bb462807851"
format_stamp: "Formatted at 2026-08-06 16:07:02 on dist-test-slave-7z32"
server_key: "3389b400ed4ad719a57eb308684a2280"
server_key_iv: "05b989296f6345ffbc6928988ec47d7a"
server_key_version: "encryptionkey@0"
I20260806 16:07:04.040858 21122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:04.062299 21122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:04.066956 21122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:04.067234 21122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:04.068477 21122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:04.070070 21122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:04.070144 21122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:04.070202 21122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:04.070333 21122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:04.085129 21122 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:42381
I20260806 16:07:04.085151 21300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:42381 every 8 connection(s)
I20260806 16:07:04.086738 21122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:07:04.087860 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 21122
Aug 06 16:07:04 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:04.101099 20929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:04.088989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54544 (local address 127.8.49.62:45221)
0806 16:07:04.089349 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:04.089355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:04.089560 (+ 205us) server_negotiation.cc:407] Connection header received
0806 16:07:04.089674 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:04.089677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:04.089734 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:04.089915 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:04.091056 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.091746 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.092907 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.093265 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.093387 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:04.095559 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:04.095567 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:04.095590 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:07:04.095636 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:04.098203 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:04.098870 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:04.098880 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:04.098889 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:04.099060 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:04.099414 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:04.099420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:04.099426 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:04.099979 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:04.100253 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:04.100584 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 16:07:04.100938 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239}
Aug 06 16:07:04 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:07:04.104243 21306 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:04.090560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:59647 (local address 127.8.49.62:45221)
0806 16:07:04.090898 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:04.090926 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:04.092320 (+ 1394us) server_negotiation.cc:407] Connection header received
0806 16:07:04.093602 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:04.093607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:04.093738 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:04.093883 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:04.095787 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.096675 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.098443 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.098722 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.098892 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:04.104031 (+ 5139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:59647: BlockingRecv error: recv got EOF from 127.8.49.3:59647 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":132,"threads_started":1}
I20260806 16:07:04.105388 21307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:04.091050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:59647)
0806 16:07:04.092150 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:04.092213 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:04.093300 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:04.094219 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:04.094232 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:04.094930 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:04.095605 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:04.095625 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.096977 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.096984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:04.098205 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:04.098221 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.098971 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.098976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:04.099194 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:04.099886 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:04.099906 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:04.103802 (+ 3896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":186,"threads_started":1}
W20260806 16:07:04.106146 21302 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:45221 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.49.62:45221: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Aug 06 16:07:04 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:04 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:04.113780 21307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:04.106667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:41289)
0806 16:07:04.106965 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:04.106979 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:04.107070 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:04.107521 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:04.107524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:04.107697 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:04.107820 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:04.107826 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.108900 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.108903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:04.109652 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:04.109665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.110297 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.110301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:04.110418 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:04.111239 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:04.111254 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:04.113652 (+ 2398us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220}
W20260806 16:07:04.114310 21306 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:04.106802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:41289 (local address 127.8.49.62:45221)
0806 16:07:04.107025 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:04.107030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:04.107047 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:07:04.107251 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:04.107254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:04.107311 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:04.107402 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:04.107977 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.108726 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.109830 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.110109 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.110255 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:04.114127 (+ 3872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:41289: BlockingRecv error: recv got EOF from 127.8.49.3:41289 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 06 16:07:04 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:04 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:04.121604 21307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:04.114441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:35277)
0806 16:07:04.114693 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:04.114708 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:04.114786 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:04.115174 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:04.115177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:04.115344 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:04.115442 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:04.115447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.116612 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.116616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:04.117448 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:04.117462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.118185 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.118189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:04.118292 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:04.119128 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:04.119142 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:04.121477 (+ 2335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167}
W20260806 16:07:04.121891 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:04.114610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:35277 (local address 127.8.49.62:45221)
0806 16:07:04.114869 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:04.114877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:04.114903 (+ 26us) server_negotiation.cc:407] Connection header received
0806 16:07:04.114962 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:04.114964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:04.115020 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:04.115117 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:04.115730 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.116428 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.117633 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.117983 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.118156 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:04.121759 (+ 3603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:35277: BlockingRecv error: recv got EOF from 127.8.49.3:35277 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
W20260806 16:07:04.121996 21302 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 16:07:04 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:04 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:04.556782 21308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:04.549160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:59849)
0806 16:07:04.549948 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:04.549967 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:04.550164 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:04.550597 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:04.550601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:04.550932 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:04.551080 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:04.551087 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.551978 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.551982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:04.552649 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:04.552659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.553217 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.553221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:04.553328 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:04.553988 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:04.554001 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:04.556625 (+ 2624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":192,"threads_started":1}
W20260806 16:07:04.557157 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:04.549541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:59849 (local address 127.8.49.62:45221)
0806 16:07:04.549816 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:04.549823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:04.550081 (+ 258us) server_negotiation.cc:407] Connection header received
0806 16:07:04.550313 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:04.550316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:04.550448 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:04.550559 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:04.551222 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.551837 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.552811 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.553067 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.553199 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:04.556983 (+ 3784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:59849: BlockingRecv error: recv got EOF from 127.8.49.1:59849 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 06 16:07:04 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:04 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:04.859884 21310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:04.851742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:52387)
0806 16:07:04.852245 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:04.852262 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:04.852422 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:04.852748 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:04.852751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:04.852984 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:04.853118 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:04.853124 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.854066 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.854069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:04.854916 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:04.854932 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.855527 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.855531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:04.855610 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:04.856401 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:04.856419 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:04.859697 (+ 3278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":112,"threads_started":1}
W20260806 16:07:04.860198 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:04.852054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:52387 (local address 127.8.49.62:45221)
0806 16:07:04.852214 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:04.852218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:04.852376 (+ 158us) server_negotiation.cc:407] Connection header received
0806 16:07:04.852534 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:04.852537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:04.852592 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:04.852680 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:04.853292 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.853901 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.855122 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:04.855370 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:04.855476 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:04.860031 (+ 4555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:52387: BlockingRecv error: recv got EOF from 127.8.49.2:52387 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 16:07:05 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:05 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:05.130816 21311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:05.123138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:33135)
0806 16:07:05.123493 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:05.123514 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:05.123658 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:05.124205 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:05.124209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:05.124531 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:05.124699 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:05.124710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.125710 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.125714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:05.126521 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:05.126533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.127138 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.127141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:05.127247 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:05.127946 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:05.127961 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:05.130572 (+ 2611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":140,"threads_started":1}
W20260806 16:07:05.130931 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:05.123408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:33135 (local address 127.8.49.62:45221)
0806 16:07:05.123592 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:05.123596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:05.123617 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:07:05.123832 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:05.123835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:05.123897 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:05.124119 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:05.124837 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.125586 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.126656 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.126998 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.127122 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:05.130748 (+ 3626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:33135: BlockingRecv error: recv got EOF from 127.8.49.3:33135 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 16:07:05 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:05 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:07:05.565824 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:05.558194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:59599 (local address 127.8.49.62:45221)
0806 16:07:05.558434 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:05.558443 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:05.558785 (+ 342us) server_negotiation.cc:407] Connection header received
0806 16:07:05.559011 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:05.559016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:05.559081 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:05.559269 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:05.559854 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.560460 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.561334 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.561577 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.561701 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:05.565685 (+ 3984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:59599: BlockingRecv error: recv got EOF from 127.8.49.1:59599 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 16:07:05.565812 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:05.558083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:59599)
0806 16:07:05.558660 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:05.558677 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:05.558883 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:05.559293 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:05.559296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:05.559582 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:05.559715 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:05.559722 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.560584 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.560590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:05.561193 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:05.561202 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.561714 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.561718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:05.561814 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:05.562510 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:05.562523 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:05.565476 (+ 2953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":127,"threads_started":1}
Aug 06 16:07:05 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:05 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:05.867868 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:05.861051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:40729)
0806 16:07:05.861560 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:05.861579 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:05.861768 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:05.862108 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:05.862111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:05.862365 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:05.862489 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:05.862496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.863473 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.863478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:05.864098 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:05.864107 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.864628 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.864632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:05.864725 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:05.865394 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:05.865407 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:05.867637 (+ 2230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":164,"threads_started":1}
W20260806 16:07:05.867933 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:05.861121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:40729 (local address 127.8.49.62:45221)
0806 16:07:05.861431 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:05.861439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:05.861692 (+ 253us) server_negotiation.cc:407] Connection header received
0806 16:07:05.861895 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:05.861899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:05.861959 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:05.862074 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:05.862632 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.863331 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.864238 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:05.864484 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:05.864612 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:05.867797 (+ 3185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:40729: BlockingRecv error: recv got EOF from 127.8.49.2:40729 (error 108)
Metrics: {"server-negotiator.queue_time_us":176}
Aug 06 16:07:06 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:06 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:06.140496 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:06.132518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:60603)
0806 16:07:06.132959 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:06.132975 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:06.133117 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:06.133696 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:06.133699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:06.133950 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:06.134157 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:06.134167 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.135425 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.135429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:06.136513 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:06.136526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.137108 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.137111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:06.137193 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:06.137843 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:06.137856 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:06.140307 (+ 2451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":146,"threads_started":1}
W20260806 16:07:06.140659 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:06.132802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:60603 (local address 127.8.49.62:45221)
0806 16:07:06.133014 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:06.133019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:06.133040 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:07:06.133325 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:06.133329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:06.133399 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:06.133568 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:06.134375 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.135244 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.136682 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.136942 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.137055 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:06.140496 (+ 3441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:60603: BlockingRecv error: recv got EOF from 127.8.49.3:60603 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 16:07:06 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:06 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:06.575673 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:06.567277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:36983)
0806 16:07:06.567883 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:06.567901 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:06.568149 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:06.568604 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:06.568607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:06.568902 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:06.569070 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:06.569078 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.569975 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.569980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:06.570681 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:06.570691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.571317 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.571322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:06.571429 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:06.572247 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:06.572262 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:06.575438 (+ 3176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":147,"threads_started":1}
W20260806 16:07:06.575832 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:06.567505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:36983 (local address 127.8.49.62:45221)
0806 16:07:06.567780 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:06.567787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:06.568054 (+ 267us) server_negotiation.cc:407] Connection header received
0806 16:07:06.568284 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:06.568287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:06.568347 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:06.568577 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:06.569221 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.569831 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.570879 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.571163 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.571301 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:06.575651 (+ 4350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:36983: BlockingRecv error: recv got EOF from 127.8.49.1:36983 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 06 16:07:06 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:06 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:06.876031 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:06.869226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:45035)
0806 16:07:06.869580 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:06.869595 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:06.869744 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:06.870078 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:06.870080 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:06.870328 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:06.870465 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:06.870471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.871399 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.871411 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:06.872048 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:06.872057 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.872763 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.872766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:06.872843 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:06.873538 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:06.873551 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:06.875810 (+ 2259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":130,"threads_started":1}
W20260806 16:07:06.876286 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:06.869425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:45035 (local address 127.8.49.62:45221)
0806 16:07:06.869608 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:06.869612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:06.869740 (+ 128us) server_negotiation.cc:407] Connection header received
0806 16:07:06.869836 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:06.869839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:06.869894 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:06.870014 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:06.870595 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.871249 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.872271 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:06.872611 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:06.872725 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:06.876120 (+ 3395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:45035: BlockingRecv error: recv got EOF from 127.8.49.2:45035 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 16:07:07 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:07 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:07.148559 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:07.141764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:37933)
0806 16:07:07.142206 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:07.142223 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:07.142386 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:07.142696 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:07.142699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:07.143011 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:07.143126 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:07.143132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.143912 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.143916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:07.144521 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:07.144531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.144989 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.144993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:07.145080 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:07.145725 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:07.145748 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:07.148349 (+ 2601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":122,"threads_started":1}
W20260806 16:07:07.148728 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:07.141757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:37933 (local address 127.8.49.62:45221)
0806 16:07:07.141990 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:07.141997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:07.142317 (+ 320us) server_negotiation.cc:407] Connection header received
0806 16:07:07.142518 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:07.142520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:07.142569 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:07.142662 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:07.143248 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.143789 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.144647 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.144863 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.144976 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:07.148566 (+ 3590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:37933: BlockingRecv error: recv got EOF from 127.8.49.3:37933 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 16:07:07 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:07 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:07.584110 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:07.576982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:57895)
0806 16:07:07.577366 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:07.577388 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:07.577535 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:07.578004 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:07.578007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:07.578229 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:07.578365 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:07.578371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.579627 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.579630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:07.580512 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:07.580525 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.581155 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.581158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:07.581233 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:07.581898 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:07.581911 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:07.583962 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":115,"threads_started":1}
W20260806 16:07:07.584470 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:07.577120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:57895 (local address 127.8.49.62:45221)
0806 16:07:07.577394 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:07.577399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:07.577497 (+ 98us) server_negotiation.cc:407] Connection header received
0806 16:07:07.577737 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:07.577746 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:07.577814 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:07.577920 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:07.578539 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.579461 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.580687 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.580986 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.581088 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:07.584308 (+ 3220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:57895: BlockingRecv error: recv got EOF from 127.8.49.1:57895 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
Aug 06 16:07:07 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:07 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:07.885031 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:07.877362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:44309)
0806 16:07:07.877827 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:07.877846 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:07.878001 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:07.878469 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:07.878474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:07.878768 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:07.878999 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:07.879009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.880382 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.880386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:07.881099 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:07.881112 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.881724 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.881727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:07.881804 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:07.882504 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:07.882518 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:07.884863 (+ 2345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":141,"threads_started":1}
W20260806 16:07:07.885473 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:07.877476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:44309 (local address 127.8.49.62:45221)
0806 16:07:07.877740 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:07.877746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:07.877959 (+ 213us) server_negotiation.cc:407] Connection header received
0806 16:07:07.878155 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:07.878158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:07.878229 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:07.878400 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:07.879243 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.880164 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.881285 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:07.881552 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:07.881653 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:07.885275 (+ 3622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:44309: BlockingRecv error: recv got EOF from 127.8.49.2:44309 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 06 16:07:08 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:08 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:08.157732 21321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:08.149965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:39567)
0806 16:07:08.150373 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:08.150389 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:08.150505 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:08.151005 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:08.151009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:08.151263 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:08.151386 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:08.151392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.152489 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.152493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:08.153327 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:08.153341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.154093 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.154097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:08.154209 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:08.155321 (+ 1112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:08.155338 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:08.157596 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":147,"threads_started":1}
W20260806 16:07:08.158222 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:08.150189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:39567 (local address 127.8.49.62:45221)
0806 16:07:08.150459 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:08.150464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:08.150491 (+ 27us) server_negotiation.cc:407] Connection header received
0806 16:07:08.150640 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:08.150643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:08.150720 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:08.150953 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:08.151586 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.152302 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.153574 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.153922 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.154033 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:08.157907 (+ 3874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:39567: BlockingRecv error: recv got EOF from 127.8.49.3:39567 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
Aug 06 16:07:08 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:08 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:08.602614 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:08.585486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:35721)
0806 16:07:08.585836 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:08.585862 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:08.585983 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:08.586379 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:08.586381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:08.586606 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:08.586728 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:08.586733 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.587788 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.587795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:08.588513 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:08.588523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.589031 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.589035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:08.589119 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:08.599647 (+ 10528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:08.599668 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:08.602389 (+ 2721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":145,"threads_started":1}
W20260806 16:07:08.603061 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:08.585699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:35721 (local address 127.8.49.62:45221)
0806 16:07:08.585916 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:08.585920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:08.585943 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:07:08.586093 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:08.586096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:08.586154 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:08.586363 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:08.587088 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.587658 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.588647 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.588883 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.589009 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:08.602810 (+ 13801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:35721: BlockingRecv error: recv got EOF from 127.8.49.1:35721 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 16:07:08 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:08 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:08.893445 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:08.886281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:39163)
0806 16:07:08.886740 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:08.886759 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:08.886964 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:08.887304 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:08.887307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:08.887564 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:08.887665 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:08.887672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.888572 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.888577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:08.889245 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:08.889255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.889799 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.889803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:08.889902 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:08.890615 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:08.890627 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:08.893276 (+ 2649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":151,"threads_started":1}
W20260806 16:07:08.893630 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:08.886529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:39163 (local address 127.8.49.62:45221)
0806 16:07:08.886713 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:08.886719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:08.886883 (+ 164us) server_negotiation.cc:407] Connection header received
0806 16:07:08.887105 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:08.887110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:08.887165 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:08.887276 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:08.887821 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.888446 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.889393 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:08.889655 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:08.889772 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:08.893472 (+ 3700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:39163: BlockingRecv error: recv got EOF from 127.8.49.2:39163 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 16:07:09 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:09 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:09.166497 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:09.159045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:37067)
0806 16:07:09.159613 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:09.159630 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:09.159827 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:09.160155 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:09.160158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:09.160403 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:09.160504 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:09.160510 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.161341 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.161346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:09.161995 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:09.162006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.162787 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.162791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:09.162928 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:09.163634 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:09.163658 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:09.166354 (+ 2696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":159,"threads_started":1}
W20260806 16:07:09.166749 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:09.159259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:37067 (local address 127.8.49.62:45221)
0806 16:07:09.159537 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:09.159543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:09.159742 (+ 199us) server_negotiation.cc:407] Connection header received
0806 16:07:09.159975 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:09.159978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:09.160033 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:09.160127 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:09.160636 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.161211 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.162213 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.162574 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.162780 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:09.166641 (+ 3861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:37067: BlockingRecv error: recv got EOF from 127.8.49.3:37067 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
Aug 06 16:07:09 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:09 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:09.611711 21326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:09.603842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:33493)
0806 16:07:09.604301 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:09.604318 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:09.604464 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:09.604771 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:09.604774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:09.605035 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:09.605167 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:09.605173 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.606067 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.606079 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:09.606721 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:09.606730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.607528 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.607531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:09.607614 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:09.608336 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:09.608349 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:09.611549 (+ 3200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":149,"threads_started":1}
W20260806 16:07:09.612241 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:09.604043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:33493 (local address 127.8.49.62:45221)
0806 16:07:09.604210 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:09.604216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:09.604470 (+ 254us) server_negotiation.cc:407] Connection header received
0806 16:07:09.604546 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:09.604548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:09.604613 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:09.604718 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:09.605352 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.605908 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.607005 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.607243 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.607355 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:09.611804 (+ 4449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:33493: BlockingRecv error: recv got EOF from 127.8.49.1:33493 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 16:07:09 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:09 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:09.902585 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:09.895023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:51281)
0806 16:07:09.895368 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:09.895387 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:09.895561 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:09.896050 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:09.896053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:09.896380 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:09.896539 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:09.896547 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.897739 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.897744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:09.898377 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:09.898386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.898963 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.898968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:09.899055 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:09.899844 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:09.899858 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:09.902380 (+ 2522us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":120,"threads_started":1}
W20260806 16:07:09.902734 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:09.895155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:51281 (local address 127.8.49.62:45221)
0806 16:07:09.895400 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:09.895405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:09.895517 (+ 112us) server_negotiation.cc:407] Connection header received
0806 16:07:09.895720 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:09.895725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:09.895799 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:09.895949 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:09.896704 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.897605 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.898524 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:09.898781 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:09.898963 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:09.902614 (+ 3651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:51281: BlockingRecv error: recv got EOF from 127.8.49.2:51281 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Aug 06 16:07:10 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:10 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:10.176930 21328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:10.167719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:54529)
0806 16:07:10.168224 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:10.168252 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:10.168532 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:10.169033 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:10.169037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:10.169344 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:10.169492 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:10.169500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.170531 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.170535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:10.171550 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:10.171566 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.172433 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.172438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:10.172556 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:10.173443 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:10.173459 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:10.176696 (+ 3237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":109,"threads_started":1}
W20260806 16:07:10.177129 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:10.168128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:54529 (local address 127.8.49.62:45221)
0806 16:07:10.168341 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:10.168349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:10.168412 (+ 63us) server_negotiation.cc:407] Connection header received
0806 16:07:10.168683 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:10.168686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:10.168745 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:10.168840 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:10.169709 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.170346 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.171788 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.172108 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.172216 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:10.176952 (+ 4736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:54529: BlockingRecv error: recv got EOF from 127.8.49.3:54529 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 16:07:10 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:10 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:10.620126 21332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:10.612955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:55961)
0806 16:07:10.613540 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:10.613576 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:10.613781 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:10.614139 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:10.614142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:10.614392 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:10.614503 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:10.614509 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.615400 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.615405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:10.615992 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:10.616001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.616489 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.616493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:10.616578 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:10.617239 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:10.617253 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:10.619949 (+ 2696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":180,"threads_started":1}
W20260806 16:07:10.620591 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:10.613230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:55961 (local address 127.8.49.62:45221)
0806 16:07:10.613541 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:10.613548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:10.613698 (+ 150us) server_negotiation.cc:407] Connection header received
0806 16:07:10.613908 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:10.613911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:10.613978 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:10.614097 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:10.614650 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.615278 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.616128 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.616348 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.616473 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:10.620418 (+ 3945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:55961: BlockingRecv error: recv got EOF from 127.8.49.1:55961 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 06 16:07:10 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:10 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:10.913247 21334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:10.904095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:55009)
0806 16:07:10.904585 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:10.904602 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:10.904816 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:10.905396 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:10.905400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:10.905781 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:10.905994 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:10.906007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.907016 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.907023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:10.907974 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:10.907988 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.908647 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.908653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:10.908749 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:10.909550 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:10.909564 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:10.912871 (+ 3307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":118,"threads_started":1}
W20260806 16:07:10.913291 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:10.904268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:55009 (local address 127.8.49.62:45221)
0806 16:07:10.904506 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:10.904512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:10.904700 (+ 188us) server_negotiation.cc:407] Connection header received
0806 16:07:10.905016 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:10.905019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:10.905108 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:10.905213 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:10.906183 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.906778 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.908166 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:10.908430 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:10.908550 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:10.913143 (+ 4593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:55009: BlockingRecv error: recv got EOF from 127.8.49.2:55009 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 06 16:07:11 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:11 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:11.187879 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:11.178627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:33989)
0806 16:07:11.179303 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:11.179321 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:11.179716 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:11.180426 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:11.180429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:11.180758 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:11.180912 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:11.180920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.181882 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.181889 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:11.182515 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:11.182528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.183514 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.183517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:11.183663 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:11.184439 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:11.184453 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:11.187678 (+ 3225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":154,"threads_started":1}
W20260806 16:07:11.188493 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:11.178661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:33989 (local address 127.8.49.62:45221)
0806 16:07:11.178978 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:11.178986 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:11.179702 (+ 716us) server_negotiation.cc:407] Connection header received
0806 16:07:11.179999 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:11.180004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:11.180088 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:11.180357 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:11.181098 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.181752 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.182769 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.183327 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.183445 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:11.188289 (+ 4844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:33989: BlockingRecv error: recv got EOF from 127.8.49.3:33989 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 06 16:07:11 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:11 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:11.629710 21336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:11.621556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:58897)
0806 16:07:11.622068 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:11.622097 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:11.622287 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:11.622963 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:11.622967 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:11.623381 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:11.623595 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:11.623606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.624760 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.624764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:11.625402 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:11.625413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.626001 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.626013 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:11.626093 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:11.626758 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:11.626772 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:11.629556 (+ 2784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":120,"threads_started":1}
W20260806 16:07:11.630066 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:11.622001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:58897 (local address 127.8.49.62:45221)
0806 16:07:11.622207 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:11.622211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:11.622232 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:07:11.622577 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:11.622582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:11.622664 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:11.622805 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:11.623824 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.624613 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.625561 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.625850 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.625963 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:11.629884 (+ 3921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:58897: BlockingRecv error: recv got EOF from 127.8.49.1:58897 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 06 16:07:11 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:11 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:11.922330 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:11.914955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:51381)
0806 16:07:11.915371 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:11.915393 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:11.915575 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:11.916003 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:11.916007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:11.916322 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:11.916610 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:11.916620 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.917799 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.917802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:11.918476 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:11.918486 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.919050 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.919053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:11.919158 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:11.919815 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:11.919829 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:11.922164 (+ 2335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":124,"threads_started":1}
W20260806 16:07:11.922609 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:11.915080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:51381 (local address 127.8.49.62:45221)
0806 16:07:11.915371 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:11.915376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:11.915514 (+ 138us) server_negotiation.cc:407] Connection header received
0806 16:07:11.915714 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:11.915718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:11.915791 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:11.915894 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:11.916785 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.917652 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.918633 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:11.918903 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:11.919010 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:11.922402 (+ 3392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:51381: BlockingRecv error: recv got EOF from 127.8.49.2:51381 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 06 16:07:12 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:12 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:12.197461 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:12.189500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:51597)
0806 16:07:12.189868 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:12.189889 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:12.190020 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:12.190608 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:12.190612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:12.190927 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:12.191058 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:12.191067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.192218 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.192221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:12.192836 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:12.192845 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.193471 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.193476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:12.193577 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:12.194574 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:12.194591 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:12.197330 (+ 2739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":131,"threads_started":1}
W20260806 16:07:12.197832 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:12.189641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:51597 (local address 127.8.49.62:45221)
0806 16:07:12.189861 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:12.189866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:12.189992 (+ 126us) server_negotiation.cc:407] Connection header received
0806 16:07:12.190220 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:12.190225 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:12.190311 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:12.190455 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:12.191289 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.192044 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.192991 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.193280 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.193406 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:12.197640 (+ 4234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:51597: BlockingRecv error: recv got EOF from 127.8.49.3:51597 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 06 16:07:12 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:12 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:12.638281 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:12.631027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:58509)
0806 16:07:12.631579 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:12.631598 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:12.631771 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:12.632316 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:12.632319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:12.632557 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:12.632680 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:12.632686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.633635 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.633638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:12.634244 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:12.634253 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.634781 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.634785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:12.635036 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:12.635677 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:12.635689 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:12.638135 (+ 2446us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":280,"threads_started":1}
W20260806 16:07:12.638592 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:12.631764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:58509 (local address 127.8.49.62:45221)
0806 16:07:12.631967 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:12.631971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:12.631989 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:07:12.632041 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:12.632044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:12.632113 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:12.632242 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:12.632843 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.633489 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.634381 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.634626 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.634735 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:12.638396 (+ 3661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:58509: BlockingRecv error: recv got EOF from 127.8.49.1:58509 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 06 16:07:12 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:12 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:12.931097 21340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:12.923697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:33281)
0806 16:07:12.924080 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:12.924097 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:12.924207 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:12.924620 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:12.924623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:12.924834 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:12.924982 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:12.924988 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.926038 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.926042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:12.926906 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:12.926920 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.927580 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.927587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:12.927686 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:12.928300 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:12.928314 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:12.930833 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":133,"threads_started":1}
W20260806 16:07:12.931135 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:12.923958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:33281 (local address 127.8.49.62:45221)
0806 16:07:12.924132 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:12.924137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:12.924158 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:07:12.924368 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:12.924371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:12.924434 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:12.924553 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:12.925153 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.925837 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.927065 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:12.927401 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:12.927530 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:12.931012 (+ 3482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:33281: BlockingRecv error: recv got EOF from 127.8.49.2:33281 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 16:07:13 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:13 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:13.212682 21344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:13.198668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:50475)
0806 16:07:13.199112 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:13.199132 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:13.199272 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:13.199638 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:13.199641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:13.199885 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:13.200025 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:13.200031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.201045 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.201048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:13.201741 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:13.201753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.202313 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.202316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:13.202392 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:13.209116 (+ 6724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:13.209167 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:13.212432 (+ 3265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":198,"threads_started":1}
W20260806 16:07:13.213007 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:13.198732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:50475 (local address 127.8.49.62:45221)
0806 16:07:13.198978 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:13.198984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:13.199242 (+ 258us) server_negotiation.cc:407] Connection header received
0806 16:07:13.199388 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:13.199390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:13.199454 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:13.199545 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:13.200207 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.200865 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.201911 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.202151 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.202306 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:13.212812 (+ 10506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:50475: BlockingRecv error: recv got EOF from 127.8.49.3:50475 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Aug 06 16:07:13 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:13 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:13.654809 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:13.639684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:40237)
0806 16:07:13.640320 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:13.640347 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:13.640540 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:13.640915 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:13.640919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:13.641141 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:13.641253 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:13.641259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.642154 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.642157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:13.642903 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:13.642916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.643549 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.643552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:13.643623 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:13.652078 (+ 8455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:13.652107 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:13.654637 (+ 2530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":150,"threads_started":1}
W20260806 16:07:13.655061 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:13.639700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:40237 (local address 127.8.49.62:45221)
0806 16:07:13.639925 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:13.639930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:13.640535 (+ 605us) server_negotiation.cc:407] Connection header received
0806 16:07:13.640673 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:13.640676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:13.640738 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:13.640815 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:13.641415 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.642008 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.643053 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.643375 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.643480 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:13.654867 (+ 11387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:40237: BlockingRecv error: recv got EOF from 127.8.49.1:40237 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 16:07:13 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:13 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:13.940706 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:13.932548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:54391)
0806 16:07:13.932914 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:13.932943 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:13.933072 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:13.933569 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:13.933572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:13.933800 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:13.933922 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:13.933929 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.935104 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.935110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:13.936122 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:13.936137 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.936830 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.936834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:13.936929 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:13.937721 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:13.937733 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:13.940545 (+ 2812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":143,"threads_started":1}
W20260806 16:07:13.941032 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:13.932794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:54391 (local address 127.8.49.62:45221)
0806 16:07:13.932999 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:13.933003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:13.933027 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:07:13.933280 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:13.933284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:13.933350 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:13.933486 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:13.934116 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.934939 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.936292 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:13.936644 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:13.936768 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:13.940801 (+ 4033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:54391: BlockingRecv error: recv got EOF from 127.8.49.2:54391 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 06 16:07:14 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:14 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 16:07:14.222721 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:14.214231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:57817 (local address 127.8.49.62:45221)
0806 16:07:14.214423 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:14.214427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:14.214444 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:07:14.214604 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:14.214607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:14.214665 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:14.214767 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:14.215738 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.216355 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.217439 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.217765 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.217878 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:14.222615 (+ 4737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:57817: BlockingRecv error: recv got EOF from 127.8.49.3:57817 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 16:07:14.222697 21347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:14.213998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:57817)
0806 16:07:14.214332 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:14.214357 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:14.214483 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:14.215168 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:14.215171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:14.215406 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:14.215525 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:14.215531 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.216584 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.216588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:14.217262 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:14.217272 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.217979 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.217983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:14.218090 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:14.218968 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:14.218981 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:14.222492 (+ 3511us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":136,"threads_started":1}
Aug 06 16:07:14 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:14 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:14.664647 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:14.656172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:46265)
0806 16:07:14.656640 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:14.656662 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:14.656860 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:14.657283 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:14.657288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:14.657548 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:14.657686 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:14.657692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.658967 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.658970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:14.659770 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:14.659782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.660314 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.660318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:14.660408 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:14.661399 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:14.661413 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:14.664459 (+ 3046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":222,"threads_started":1}
W20260806 16:07:14.664990 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:14.656437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:46265 (local address 127.8.49.62:45221)
0806 16:07:14.656732 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:14.656738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:14.656770 (+ 32us) server_negotiation.cc:407] Connection header received
0806 16:07:14.657012 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:14.657018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:14.657093 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:14.657214 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:14.657911 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.658778 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.659911 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.660150 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.660246 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:14.664786 (+ 4540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:46265: BlockingRecv error: recv got EOF from 127.8.49.1:46265 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
Aug 06 16:07:14 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:14 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:14.949120 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:14.941937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:42377)
0806 16:07:14.942443 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:14.942463 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:14.942613 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:14.943031 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:14.943034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:14.943304 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:14.943422 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:14.943429 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.944395 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.944399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:14.945169 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:14.945182 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.945894 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.945898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:14.945974 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:14.946741 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:14.946753 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:14.948941 (+ 2188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":116,"threads_started":1}
W20260806 16:07:14.949456 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:14.942104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:42377 (local address 127.8.49.62:45221)
0806 16:07:14.942333 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:14.942340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:14.942573 (+ 233us) server_negotiation.cc:407] Connection header received
0806 16:07:14.942753 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:14.942756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:14.942865 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:14.942983 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:14.943566 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.944174 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.945422 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:14.945726 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:14.945848 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:14.949286 (+ 3438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:42377: BlockingRecv error: recv got EOF from 127.8.49.2:42377 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 16:07:15 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:15 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:15.232867 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:15.224221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:46029)
0806 16:07:15.224784 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:15.224805 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:15.224995 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:15.225436 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:15.225439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:15.225756 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:15.225897 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:15.225905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.227018 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.227025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:15.227841 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:15.227855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.228650 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.228655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:15.228771 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:15.229590 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:15.229605 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:15.232659 (+ 3054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":140,"threads_started":1}
W20260806 16:07:15.232975 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:15.224549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:46029 (local address 127.8.49.62:45221)
0806 16:07:15.224833 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:15.224838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:15.224924 (+ 86us) server_negotiation.cc:407] Connection header received
0806 16:07:15.225176 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:15.225180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:15.225251 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:15.225360 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:15.226105 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.226738 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.228030 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.228301 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.228438 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:15.232856 (+ 4418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:46029: BlockingRecv error: recv got EOF from 127.8.49.3:46029 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
Aug 06 16:07:15 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:15 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:15.674360 21352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:15.666053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:35521)
0806 16:07:15.666495 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:15.666513 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:15.666664 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:15.667116 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:15.667120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:15.667366 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:15.667489 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:15.667496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.668618 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.668625 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:15.669780 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:15.669799 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.670576 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.670581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:15.670710 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:15.671618 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:15.671639 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:15.674148 (+ 2509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":131,"threads_started":1}
W20260806 16:07:15.674564 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:15.666564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:35521 (local address 127.8.49.62:45221)
0806 16:07:15.666776 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:15.666784 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:15.666804 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:07:15.666916 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:15.666918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:15.666968 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:15.667084 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:15.667631 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.668397 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.669981 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.670364 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.670496 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:15.674359 (+ 3863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:35521: BlockingRecv error: recv got EOF from 127.8.49.1:35521 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 06 16:07:15 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:15 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:15.956945 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:15.950236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:45579)
0806 16:07:15.950664 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:15.950686 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:15.950888 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:15.951225 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:15.951229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:15.951473 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:15.951579 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:15.951586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.952355 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.952360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:15.952915 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:15.952924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.953408 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.953411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:15.953499 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:15.954181 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:15.954193 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:15.956767 (+ 2574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":105,"threads_started":1}
W20260806 16:07:15.957130 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:15.950464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:45579 (local address 127.8.49.62:45221)
0806 16:07:15.950663 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:15.950668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:15.950792 (+ 124us) server_negotiation.cc:407] Connection header received
0806 16:07:15.951033 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:15.951037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:15.951099 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:15.951193 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:15.951717 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.952242 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.953051 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:15.953275 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:15.953385 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:15.956970 (+ 3585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:45579: BlockingRecv error: recv got EOF from 127.8.49.2:45579 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 16:07:16 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:16 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:16.243808 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:16.234425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:57323)
0806 16:07:16.235042 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:16.235068 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:16.235280 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:16.235733 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:16.235737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:16.236145 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:16.236318 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:16.236326 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.237713 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.237718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:16.238702 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:16.238717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.239596 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.239600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:16.239680 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:16.240437 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:16.240451 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:16.243595 (+ 3144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":210,"threads_started":1}
W20260806 16:07:16.243940 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:16.234636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:57323 (local address 127.8.49.62:45221)
0806 16:07:16.234818 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:16.234853 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:16.235251 (+ 398us) server_negotiation.cc:407] Connection header received
0806 16:07:16.235477 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:16.235483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:16.235544 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:16.235655 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:16.236692 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.237531 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.238927 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.239365 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.239506 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:16.243799 (+ 4293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:57323: BlockingRecv error: recv got EOF from 127.8.49.3:57323 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 06 16:07:16 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:16 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:16.682539 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:16.675750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:51105)
0806 16:07:16.676201 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:16.676222 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:16.676354 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:16.676718 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:16.676723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:16.677076 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:16.677241 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:16.677249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.678132 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.678138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:16.678793 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:16.678802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.679387 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.679391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:16.679480 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:16.680218 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:16.680231 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:16.682339 (+ 2108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":146,"threads_started":1}
W20260806 16:07:16.682657 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:16.675785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:51105 (local address 127.8.49.62:45221)
0806 16:07:16.675979 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:16.675985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:16.676333 (+ 348us) server_negotiation.cc:407] Connection header received
0806 16:07:16.676484 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:16.676488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:16.676565 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:16.676664 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:16.677388 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.677999 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.678967 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.679218 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.679365 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:16.682509 (+ 3144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:51105: BlockingRecv error: recv got EOF from 127.8.49.1:51105 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 16:07:16 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:16 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:16.965685 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:16.958079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:34039)
0806 16:07:16.958551 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:16.958564 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:16.958722 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:16.959199 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:16.959202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:16.959435 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:16.959534 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:16.959540 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.960322 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.960326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:16.960914 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:16.960924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.961401 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.961405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:16.961491 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:16.962208 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:16.962227 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:16.965454 (+ 3227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":133,"threads_started":1}
W20260806 16:07:16.965813 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:16.958263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:34039 (local address 127.8.49.62:45221)
0806 16:07:16.958470 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:16.958475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:16.958647 (+ 172us) server_negotiation.cc:407] Connection header received
0806 16:07:16.958983 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:16.958987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:16.959040 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:16.959172 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:16.959659 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.960204 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.961049 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:16.961275 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:16.961390 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:16.965669 (+ 4279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:34039: BlockingRecv error: recv got EOF from 127.8.49.2:34039 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 06 16:07:17 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:17 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:17.253615 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:17.245213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:48291)
0806 16:07:17.245801 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:17.245822 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:17.246072 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:17.246391 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:17.246394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:17.246631 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:17.246737 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:17.246742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.247823 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.247828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:17.248955 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:17.248980 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.249748 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.249798 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:17.249906 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:17.250602 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:17.250616 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:17.253443 (+ 2827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":234,"threads_started":1}
W20260806 16:07:17.254042 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:17.245521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:48291 (local address 127.8.49.62:45221)
0806 16:07:17.245789 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:17.245795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:17.245938 (+ 143us) server_negotiation.cc:407] Connection header received
0806 16:07:17.246198 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:17.246201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:17.246258 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:17.246358 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:17.246941 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.247600 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.249139 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.249598 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.249706 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:17.253863 (+ 4157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:48291: BlockingRecv error: recv got EOF from 127.8.49.3:48291 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 06 16:07:17 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:17 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:17.692094 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:17.683962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:53229)
0806 16:07:17.684476 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:17.684496 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:17.684697 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:17.685100 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:17.685104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:17.685393 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:17.685519 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:17.685526 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.686459 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.686464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:17.687230 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:17.687243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.687832 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.687836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:17.687928 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:17.689051 (+ 1123us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:17.689076 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:17.691914 (+ 2838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":118,"threads_started":1}
W20260806 16:07:17.692349 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:17.684018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:53229 (local address 127.8.49.62:45221)
0806 16:07:17.684420 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:17.684427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:17.684612 (+ 185us) server_negotiation.cc:407] Connection header received
0806 16:07:17.684885 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:17.684889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:17.684956 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:17.685064 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:17.685661 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.686318 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.687407 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.687680 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.687817 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:17.692171 (+ 4354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:53229: BlockingRecv error: recv got EOF from 127.8.49.1:53229 (error 108)
Metrics: {"server-negotiator.queue_time_us":258}
Aug 06 16:07:17 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:17 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:17.975100 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:17.967467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:45545)
0806 16:07:17.968099 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:17.968120 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:17.968315 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:17.968735 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:17.968737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:17.969007 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:17.969160 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:17.969171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.970196 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.970201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:17.970888 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:17.970898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.971537 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.971540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:17.971613 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:17.972236 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:17.972248 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:17.974908 (+ 2660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":192,"threads_started":1}
W20260806 16:07:17.975571 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:17.967716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:45545 (local address 127.8.49.62:45221)
0806 16:07:17.967935 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:17.967939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:17.968351 (+ 412us) server_negotiation.cc:407] Connection header received
0806 16:07:17.968449 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:17.968452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:17.968525 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:17.968686 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:17.969301 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.970047 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.971021 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:17.971331 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:17.971465 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:17.975382 (+ 3917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:45545: BlockingRecv error: recv got EOF from 127.8.49.2:45545 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 06 16:07:18 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:18 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.3@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:18.262420 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:18.255109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.3:43999)
0806 16:07:18.255417 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:18.255432 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:18.255547 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:18.256107 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:18.256117 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:18.256333 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:18.256459 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:18.256465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.257527 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.257530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:18.258210 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:18.258221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.258853 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.258857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:18.258944 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:18.259679 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:18.259693 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:18.262244 (+ 2551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":104,"threads_started":1}
W20260806 16:07:18.262810 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:18.255220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:43999 (local address 127.8.49.62:45221)
0806 16:07:18.255673 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:18.255678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:18.255712 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:07:18.255787 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:18.255791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:18.255876 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:18.256010 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:18.256700 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.257335 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.258408 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.258679 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.258787 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:18.262563 (+ 3776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.3:43999: BlockingRecv error: recv got EOF from 127.8.49.3:43999 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 16:07:18 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:18 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.1@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:18.706117 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:18.693453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.1:57153)
0806 16:07:18.693876 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:18.693896 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:18.694047 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:18.694402 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:18.694405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:18.694766 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:18.695016 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:18.695024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.696044 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.696047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:18.696713 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:18.696723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.697311 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.697314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:18.697395 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:18.701945 (+ 4550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:18.701968 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:18.705873 (+ 3905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":95,"threads_started":1}
W20260806 16:07:18.706377 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:18.693621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:57153 (local address 127.8.49.62:45221)
0806 16:07:18.693887 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:18.693892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:18.694057 (+ 165us) server_negotiation.cc:407] Connection header received
0806 16:07:18.694148 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:18.694150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:18.694204 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:18.694301 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:18.695227 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.695881 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.696877 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.697125 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.697235 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:18.706158 (+ 8923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.1:57153: BlockingRecv error: recv got EOF from 127.8.49.1:57153 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
Aug 06 16:07:18 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 16:07:18 dist-test-slave-7z32 krb5kdc[19892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.49.2@KRBTEST.COM for oryx/127.8.49.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 16:07:18.985224 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:18.976579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45221 (local address 127.8.49.2:44693)
0806 16:07:18.976901 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:18.976943 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:18.977102 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:18.977623 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:18.977626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:18.977840 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:18.977959 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:18.977965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.979104 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.979108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:18.979753 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:18.979764 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.980373 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.980377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:18.980546 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:18.981545 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:18.981586 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:18.985074 (+ 3488us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.49.62:45221: Server oryx/127.8.49.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":97,"threads_started":1}
W20260806 16:07:18.985809 20929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:18.976686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:44693 (local address 127.8.49.62:45221)
0806 16:07:18.976878 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:18.976884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:18.977043 (+ 159us) server_negotiation.cc:407] Connection header received
0806 16:07:18.977332 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:18.977337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:18.977435 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:18.977556 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:18.978178 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.978901 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.979922 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:18.980189 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:18.980297 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:18.985412 (+ 5115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.49.2:44693: BlockingRecv error: recv got EOF from 127.8.49.2:44693 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 16:07:19.093866 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 20718
I20260806 16:07:19.113809 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 20933
I20260806 16:07:19.126196 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 21122
I20260806 16:07:19.149979 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 20617
2026-08-06T16:07:19Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (19313 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T16:07:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:07:19Z Disabled control of system clock
I20260806 16:07:19.203845 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:38613
--webserver_interface=127.8.49.62
--webserver_port=0
--builtin_ntp_servers=127.8.49.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:38613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 16:07:19.350755 21373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:19.351279 21373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:19.351341 21373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:19.357208 21373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:07:19.357811 21373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:19.357915 21373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:19.357951 21373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:07:19.357982 21373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:07:19.364583 21373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:38613
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:38613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:19.365945 21373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:19.367638 21373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:19.381851 21382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:19.381851 21383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:19.382097 21385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:19.383028 21373 server_base.cc:1034] running on GCE node
I20260806 16:07:19.383841 21373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:19.385382 21373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:19.388878 21373 hybrid_clock.cc:648] HybridClock initialized: now 1786032439388770 us; error 99 us; skew 500 ppm
I20260806 16:07:19.395323 21373 webserver.cc:460] Webserver started at http://127.8.49.62:34077/ using document root <none> and password file <none>
I20260806 16:07:19.397687 21373 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:19.398191 21373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:19.398618 21373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:19.402907 21373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "015110ca677144cfa7938d9d2935afb3"
format_stamp: "Formatted at 2026-08-06 16:07:19 on dist-test-slave-7z32"
server_key: "062ae0bd721a69ba4beadc07a061d3ce"
server_key_iv: "13fea83ea579d35716fd30059c52898a"
server_key_version: "encryptionkey@0"
I20260806 16:07:19.404500 21373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "015110ca677144cfa7938d9d2935afb3"
format_stamp: "Formatted at 2026-08-06 16:07:19 on dist-test-slave-7z32"
server_key: "062ae0bd721a69ba4beadc07a061d3ce"
server_key_iv: "13fea83ea579d35716fd30059c52898a"
server_key_version: "encryptionkey@0"
I20260806 16:07:19.411111 21373 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 16:07:19.415727 21392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:19.418036 21373 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:07:19.418303 21373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "015110ca677144cfa7938d9d2935afb3"
format_stamp: "Formatted at 2026-08-06 16:07:19 on dist-test-slave-7z32"
server_key: "062ae0bd721a69ba4beadc07a061d3ce"
server_key_iv: "13fea83ea579d35716fd30059c52898a"
server_key_version: "encryptionkey@0"
I20260806 16:07:19.418510 21373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:19.446240 21373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:19.447296 21373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:19.447769 21373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:19.457448 21373 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:38613
I20260806 16:07:19.457733 21462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:38613 every 8 connection(s)
I20260806 16:07:19.458874 21373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:07:19.464021 21464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:07:19.465634 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 21373
I20260806 16:07:19.465824 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:07:19.466213 8388 external_mini_cluster.cc:1428] Setting key 2c00ca975830439061c0f62d8a4bf9e4
I20260806 16:07:19.476496 21464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3: Bootstrap starting.
I20260806 16:07:19.480837 21464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3: Neither blocks nor log segments found. Creating new log.
I20260806 16:07:19.481719 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:19.468894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.8.49.62:38613)
0806 16:07:19.470385 (+ 1491us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:19.470412 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:19.470469 (+ 57us) server_negotiation.cc:407] Connection header received
0806 16:07:19.471641 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:19.471666 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:19.472207 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:19.472581 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:19.473277 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:19.474400 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:19.476543 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:19.477328 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:19.477532 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:19.479009 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:19.479017 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:19.479044 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:07:19.479595 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:19.479695 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:07:19.479723 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:19.479833 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:19.480136 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 16:07:19.480606 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":516,"threads_started":1}
I20260806 16:07:19.482348 21464 log.cc:824] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3: Log is configured to *not* fsync() on all Append() calls
I20260806 16:07:19.485589 21464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3: No bootstrap required, opened a new log
I20260806 16:07:19.491533 21464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015110ca677144cfa7938d9d2935afb3" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38613 } }
I20260806 16:07:19.492031 21464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:07:19.492080 21464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015110ca677144cfa7938d9d2935afb3, State: Initialized, Role: FOLLOWER
I20260806 16:07:19.492803 21464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015110ca677144cfa7938d9d2935afb3" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38613 } }
I20260806 16:07:19.492977 21464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:07:19.493089 21464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:07:19.493232 21464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:07:19.494711 21464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015110ca677144cfa7938d9d2935afb3" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38613 } }
I20260806 16:07:19.495358 21464 leader_election.cc:302] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 015110ca677144cfa7938d9d2935afb3; no voters:
I20260806 16:07:19.495886 21464 leader_election.cc:288] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:07:19.496097 21474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:07:19.496474 21474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [term 1 LEADER]: Becoming Leader. State: Replica: 015110ca677144cfa7938d9d2935afb3, State: Running, Role: LEADER
I20260806 16:07:19.497004 21474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015110ca677144cfa7938d9d2935afb3" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38613 } }
I20260806 16:07:19.498066 21464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:07:19.499783 21476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "015110ca677144cfa7938d9d2935afb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015110ca677144cfa7938d9d2935afb3" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38613 } } }
I20260806 16:07:19.500053 21476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [sys.catalog]: This master's current role is: LEADER
I20260806 16:07:19.500675 21477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 015110ca677144cfa7938d9d2935afb3. Latest consensus state: current_term: 1 leader_uuid: "015110ca677144cfa7938d9d2935afb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015110ca677144cfa7938d9d2935afb3" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38613 } } }
I20260806 16:07:19.500947 21477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3 [sys.catalog]: This master's current role is: LEADER
I20260806 16:07:19.501483 21483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:07:19.505829 21483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:07:19.514063 21483 catalog_manager.cc:1347] Generated new cluster ID: 0df8871cdec8454382469b865ff3e82f
I20260806 16:07:19.514178 21483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:07:19.532842 21483 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:07:19.534035 21483 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:07:19.539196 21483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 015110ca677144cfa7938d9d2935afb3: Generated new TSK 0
I20260806 16:07:19.540382 21483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:07:19.544718 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:38613
--builtin_ntp_servers=127.8.49.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 16:07:19.698740 21498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:19.699399 21498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:19.699553 21498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:19.706138 21498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:19.706256 21498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:19.706321 21498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:07:19.712673 21498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:38613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:19.714020 21498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:19.715668 21498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:07:19.732386 21498 server_base.cc:1034] running on GCE node
W20260806 16:07:19.732463 21510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:19.732717 21509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:19.732463 21512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:19.733433 21498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:19.735118 21498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:19.738600 21498 hybrid_clock.cc:648] HybridClock initialized: now 1786032439738545 us; error 94 us; skew 500 ppm
I20260806 16:07:19.741595 21498 webserver.cc:460] Webserver started at http://127.8.49.1:34109/ using document root <none> and password file <none>
I20260806 16:07:19.742424 21498 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:19.742538 21498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:19.742870 21498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:19.745483 21498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "f25bad95b0e14a0485ef98a493120428"
format_stamp: "Formatted at 2026-08-06 16:07:19 on dist-test-slave-7z32"
server_key: "1d88114bc98e2e7dd79d8bb42777edac"
server_key_iv: "0ab810747871ef918a9442b5180967b8"
server_key_version: "encryptionkey@0"
I20260806 16:07:19.746215 21498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "f25bad95b0e14a0485ef98a493120428"
format_stamp: "Formatted at 2026-08-06 16:07:19 on dist-test-slave-7z32"
server_key: "1d88114bc98e2e7dd79d8bb42777edac"
server_key_iv: "0ab810747871ef918a9442b5180967b8"
server_key_version: "encryptionkey@0"
I20260806 16:07:19.751338 21498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:07:19.755317 21522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:19.757306 21498 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:07:19.757552 21498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "f25bad95b0e14a0485ef98a493120428"
format_stamp: "Formatted at 2026-08-06 16:07:19 on dist-test-slave-7z32"
server_key: "1d88114bc98e2e7dd79d8bb42777edac"
server_key_iv: "0ab810747871ef918a9442b5180967b8"
server_key_version: "encryptionkey@0"
I20260806 16:07:19.757725 21498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:19.786978 21498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:19.787879 21498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:19.788134 21498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:19.788872 21498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:19.790335 21498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:19.790441 21498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:19.790534 21498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:19.790596 21498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:19.805761 21498 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:37721
I20260806 16:07:19.805874 21679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:37721 every 8 connection(s)
I20260806 16:07:19.807163 21498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:07:19.816984 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 21498
I20260806 16:07:19.817198 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:07:19.817634 8388 external_mini_cluster.cc:1428] Setting key 37a23b61e3a40457fdb7a19e0d5dc786
I20260806 16:07:19.821116 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:38613
--builtin_ntp_servers=127.8.49.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 16:07:19.823058 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:19.810270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:52117 (local address 127.8.49.62:38613)
0806 16:07:19.810555 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:19.810560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:19.812552 (+ 1992us) server_negotiation.cc:407] Connection header received
0806 16:07:19.814556 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:19.814561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:19.814675 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:19.814795 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:19.816135 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:19.816800 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:19.819108 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:19.819408 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:19.819848 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:19.821855 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:19.821858 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:19.821861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:07:19.821882 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:19.821913 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:07:19.821923 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:19.822014 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:19.822656 (+ 642us) server_negotiation.cc:299] Negotiation successful
0806 16:07:19.822782 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260806 16:07:19.824713 21682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:19.810793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:38613 (local address 127.8.49.1:52117)
0806 16:07:19.812236 (+ 1443us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:19.812336 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:19.814146 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:19.815098 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:19.815107 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:19.815235 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:19.815936 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:19.815952 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:19.817256 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:19.817264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:19.818928 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:19.818946 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:19.819948 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:19.819955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:19.820404 (+ 449us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:19.821387 (+ 983us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:07:19.821389 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:19.821480 (+ 91us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:07:19.821601 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:19.822020 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:19.822034 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:19.822043 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:07:19.822621 (+ 578us) client_negotiation.cc:240] Negotiation successful
0806 16:07:19.823266 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":196,"threads_started":1}
I20260806 16:07:19.827042 21680 heartbeater.cc:347] Connected to a master server at 127.8.49.62:38613
I20260806 16:07:19.827502 21680 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:19.828383 21680 heartbeater.cc:511] Master 127.8.49.62:38613 requested a full tablet report, sending...
I20260806 16:07:19.831153 21412 ts_manager.cc:194] Registered new tserver with Master: f25bad95b0e14a0485ef98a493120428 (127.8.49.1:37721)
I20260806 16:07:19.833047 21412 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.1:52117
W20260806 16:07:19.964052 21683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:19.964517 21683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:19.964629 21683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:19.969511 21683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:19.969645 21683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:19.969748 21683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:07:19.976848 21683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:38613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:19.978593 21683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:19.980540 21683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:19.995950 21689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:19.995929 21691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:19.996515 21683 server_base.cc:1034] running on GCE node
W20260806 16:07:19.996798 21693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:19.997548 21683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:19.999003 21683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:20.002748 21683 hybrid_clock.cc:648] HybridClock initialized: now 1786032440002646 us; error 66 us; skew 500 ppm
I20260806 16:07:20.007998 21683 webserver.cc:460] Webserver started at http://127.8.49.2:40373/ using document root <none> and password file <none>
I20260806 16:07:20.009059 21683 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:20.009176 21683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:20.009521 21683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:20.013173 21683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "38d4e54a12034fe790ef17cbc82dfcb8"
format_stamp: "Formatted at 2026-08-06 16:07:20 on dist-test-slave-7z32"
server_key: "398ea9547cab50d27a03f616557a9f1d"
server_key_iv: "b1882c29a6eb7279c30a56dc1c0ddf92"
server_key_version: "encryptionkey@0"
I20260806 16:07:20.014065 21683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "38d4e54a12034fe790ef17cbc82dfcb8"
format_stamp: "Formatted at 2026-08-06 16:07:20 on dist-test-slave-7z32"
server_key: "398ea9547cab50d27a03f616557a9f1d"
server_key_iv: "b1882c29a6eb7279c30a56dc1c0ddf92"
server_key_version: "encryptionkey@0"
I20260806 16:07:20.021653 21683 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260806 16:07:20.027654 21701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:20.030455 21683 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260806 16:07:20.030685 21683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "38d4e54a12034fe790ef17cbc82dfcb8"
format_stamp: "Formatted at 2026-08-06 16:07:20 on dist-test-slave-7z32"
server_key: "398ea9547cab50d27a03f616557a9f1d"
server_key_iv: "b1882c29a6eb7279c30a56dc1c0ddf92"
server_key_version: "encryptionkey@0"
I20260806 16:07:20.031530 21683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:20.047636 21683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:20.049229 21683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:20.049777 21683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:20.051046 21683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:20.052600 21683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:20.052681 21683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:20.052794 21683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:20.052842 21683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:20.069139 21683 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:41699
I20260806 16:07:20.069177 21853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:41699 every 8 connection(s)
I20260806 16:07:20.070640 21683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:07:20.073984 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 21683
I20260806 16:07:20.074293 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:07:20.074719 8388 external_mini_cluster.cc:1428] Setting key 13a4837e56817af85029dc3c7f50b537
I20260806 16:07:20.077307 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:38613
--builtin_ntp_servers=127.8.49.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 16:07:20.085515 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:20.073799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:45073 (local address 127.8.49.62:38613)
0806 16:07:20.074082 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:20.074110 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:20.076077 (+ 1967us) server_negotiation.cc:407] Connection header received
0806 16:07:20.078216 (+ 2139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:20.078223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:20.078309 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:20.078616 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:20.080380 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:20.081008 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:20.082437 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:20.082692 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:20.082826 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:20.084622 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:20.084626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:20.084629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:07:20.084649 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:20.084678 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:07:20.084688 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:20.084788 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:20.085005 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 16:07:20.085296 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 16:07:20.087146 21857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:20.074588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:38613 (local address 127.8.49.2:45073)
0806 16:07:20.075829 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:20.075904 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:20.077782 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:20.079014 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:20.079025 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:20.079160 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:20.080134 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:20.080156 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:20.081192 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:20.081195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:20.082275 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:20.082287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:20.082941 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:20.082947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:20.083160 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:20.084118 (+ 958us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:07:20.084121 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:20.084192 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:07:20.084316 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:20.084810 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:20.084824 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:20.084836 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 16:07:20.084991 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 16:07:20.085666 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":325,"threads_started":1}
I20260806 16:07:20.089358 21854 heartbeater.cc:347] Connected to a master server at 127.8.49.62:38613
I20260806 16:07:20.089779 21854 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:20.090627 21854 heartbeater.cc:511] Master 127.8.49.62:38613 requested a full tablet report, sending...
I20260806 16:07:20.092481 21412 ts_manager.cc:194] Registered new tserver with Master: 38d4e54a12034fe790ef17cbc82dfcb8 (127.8.49.2:41699)
I20260806 16:07:20.093312 21412 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.2:45073
W20260806 16:07:20.224819 21858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:20.225203 21858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:20.225286 21858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:20.229902 21858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:20.230036 21858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:20.230147 21858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:07:20.237375 21858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:38613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:20.238942 21858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:20.240862 21858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:07:20.255103 21858 server_base.cc:1034] running on GCE node
W20260806 16:07:20.255013 21870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:20.254994 21866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:20.254956 21868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:20.256071 21858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:20.256878 21858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:20.260442 21858 hybrid_clock.cc:648] HybridClock initialized: now 1786032440260328 us; error 106 us; skew 500 ppm
I20260806 16:07:20.263968 21858 webserver.cc:460] Webserver started at http://127.8.49.3:45869/ using document root <none> and password file <none>
I20260806 16:07:20.265800 21858 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:20.265949 21858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:20.266727 21858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:20.271752 21858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "1d7a2e3ee7b842d392441f0c1505fbf5"
format_stamp: "Formatted at 2026-08-06 16:07:20 on dist-test-slave-7z32"
server_key: "c3d1cbc2fdf941aa1957a702e7dae0c2"
server_key_iv: "4893a906ece13591d0e2f75094220eeb"
server_key_version: "encryptionkey@0"
I20260806 16:07:20.272619 21858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "1d7a2e3ee7b842d392441f0c1505fbf5"
format_stamp: "Formatted at 2026-08-06 16:07:20 on dist-test-slave-7z32"
server_key: "c3d1cbc2fdf941aa1957a702e7dae0c2"
server_key_iv: "4893a906ece13591d0e2f75094220eeb"
server_key_version: "encryptionkey@0"
I20260806 16:07:20.279783 21858 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 16:07:20.284540 21879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:20.287106 21858 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:07:20.287354 21858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "1d7a2e3ee7b842d392441f0c1505fbf5"
format_stamp: "Formatted at 2026-08-06 16:07:20 on dist-test-slave-7z32"
server_key: "c3d1cbc2fdf941aa1957a702e7dae0c2"
server_key_iv: "4893a906ece13591d0e2f75094220eeb"
server_key_version: "encryptionkey@0"
I20260806 16:07:20.287566 21858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:20.303893 21858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:20.304996 21858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:20.305269 21858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:20.306139 21858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:20.307580 21858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:20.307672 21858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:20.307755 21858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:20.307808 21858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:20.321130 21858 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:45947
I20260806 16:07:20.321256 22037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:45947 every 8 connection(s)
I20260806 16:07:20.322567 21858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:07:20.330084 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 21858
I20260806 16:07:20.330241 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:07:20.330683 8388 external_mini_cluster.cc:1428] Setting key e9fbe1e8d7d36b80337d8d28cdf0cae8
I20260806 16:07:20.336781 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:20.325607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:33365 (local address 127.8.49.62:38613)
0806 16:07:20.325853 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:20.325859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:20.327653 (+ 1794us) server_negotiation.cc:407] Connection header received
0806 16:07:20.329302 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:20.329308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:20.329391 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:20.329529 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:20.331409 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:20.332139 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:20.333760 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:20.334155 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:20.334327 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:20.335985 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:20.335989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:20.335992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:07:20.336015 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:20.336052 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:07:20.336062 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:20.336158 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:20.336408 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 16:07:20.336512 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 16:07:20.337890 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:20.326345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:38613 (local address 127.8.49.3:33365)
0806 16:07:20.327444 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:20.327501 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:20.328965 (+ 1464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:20.329971 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:20.329984 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:20.330124 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:20.331017 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:20.331039 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:20.332342 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:20.332345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:20.333590 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:20.333603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:20.334399 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:20.334405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:20.334663 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:20.335575 (+ 912us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:07:20.335577 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:20.335629 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:07:20.335731 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:20.336185 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:20.336195 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:20.336200 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 16:07:20.336331 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 16:07:20.336654 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":170,"threads_started":1}
I20260806 16:07:20.340191 22039 heartbeater.cc:347] Connected to a master server at 127.8.49.62:38613
I20260806 16:07:20.340579 22039 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:20.341246 22039 heartbeater.cc:511] Master 127.8.49.62:38613 requested a full tablet report, sending...
I20260806 16:07:20.342939 21412 ts_manager.cc:194] Registered new tserver with Master: 1d7a2e3ee7b842d392441f0c1505fbf5 (127.8.49.3:45947)
I20260806 16:07:20.343844 21412 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.3:33365
I20260806 16:07:20.347116 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 16:07:20.351382 21469 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:20.349708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.8.49.62:38613)
0806 16:07:20.350064 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:20.350071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:20.350312 (+ 241us) server_negotiation.cc:407] Connection header received
0806 16:07:20.350547 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:20.350550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:20.350616 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:20.350719 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:20.351215 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56370: BlockingRecv error: recv got EOF from 127.0.0.1:56370 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 16:07:20.352277 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 21498
I20260806 16:07:20.372598 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 21683
I20260806 16:07:20.398746 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 21858
I20260806 16:07:20.419467 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 21373
2026-08-06T16:07:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1262 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T16:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:07:20Z Disabled control of system clock
I20260806 16:07:20.465588 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:33097
--webserver_interface=127.8.49.62
--webserver_port=0
--builtin_ntp_servers=127.8.49.20:43467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:33097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 16:07:20.637825 22055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:20.638401 22055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:20.638505 22055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:20.644874 22055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:07:20.645072 22055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:20.645107 22055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:20.645153 22055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:07:20.645193 22055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:07:20.653016 22055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:33097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:33097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:20.655297 22055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:20.658059 22055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:07:20.674917 22055 server_base.cc:1034] running on GCE node
W20260806 16:07:20.674770 22067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:20.674777 22065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:20.674770 22064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:20.676222 22055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:20.678299 22055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:20.681732 22055 hybrid_clock.cc:648] HybridClock initialized: now 1786032440681524 us; error 248 us; skew 500 ppm
I20260806 16:07:20.684821 22055 webserver.cc:460] Webserver started at http://127.8.49.62:45697/ using document root <none> and password file <none>
I20260806 16:07:20.685702 22055 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:20.685802 22055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:20.686040 22055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:20.688941 22055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "215994295e894902a6f27d9673442894"
format_stamp: "Formatted at 2026-08-06 16:07:20 on dist-test-slave-7z32"
server_key: "4ce990a54055c077bbf3980e9f37d7bb"
server_key_iv: "dee89b89edcf3dbcacdbce6af8bd8814"
server_key_version: "encryptionkey@0"
I20260806 16:07:20.689671 22055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "215994295e894902a6f27d9673442894"
format_stamp: "Formatted at 2026-08-06 16:07:20 on dist-test-slave-7z32"
server_key: "4ce990a54055c077bbf3980e9f37d7bb"
server_key_iv: "dee89b89edcf3dbcacdbce6af8bd8814"
server_key_version: "encryptionkey@0"
I20260806 16:07:20.695783 22055 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260806 16:07:20.699743 22078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:20.702080 22055 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:07:20.702311 22055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "215994295e894902a6f27d9673442894"
format_stamp: "Formatted at 2026-08-06 16:07:20 on dist-test-slave-7z32"
server_key: "4ce990a54055c077bbf3980e9f37d7bb"
server_key_iv: "dee89b89edcf3dbcacdbce6af8bd8814"
server_key_version: "encryptionkey@0"
I20260806 16:07:20.702462 22055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:20.708583 22055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:20.709419 22055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:20.709599 22055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:20.718413 22055 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:33097
I20260806 16:07:20.718428 22139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:33097 every 8 connection(s)
I20260806 16:07:20.719628 22055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:07:20.725128 22141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:07:20.728466 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 22055
I20260806 16:07:20.728605 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:07:20.729002 8388 external_mini_cluster.cc:1428] Setting key 66c3ba8f6a7fea5d91d9b224b51dfd91
I20260806 16:07:20.734937 22141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894: Bootstrap starting.
I20260806 16:07:20.737658 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:20.731569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.8.49.62:33097)
0806 16:07:20.732724 (+ 1155us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:20.732746 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:20.732800 (+ 54us) server_negotiation.cc:407] Connection header received
0806 16:07:20.733734 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:20.733758 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:20.734270 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:20.734780 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:20.735967 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:20.735972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:20.735989 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:07:20.735999 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:20.736075 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:07:20.736095 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:20.736300 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:20.736491 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 16:07:20.736788 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":241,"threads_started":1}
I20260806 16:07:20.739059 22141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894: Neither blocks nor log segments found. Creating new log.
I20260806 16:07:20.740295 22141 log.cc:824] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894: Log is configured to *not* fsync() on all Append() calls
I20260806 16:07:20.743877 22141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894: No bootstrap required, opened a new log
I20260806 16:07:20.748400 22141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215994295e894902a6f27d9673442894" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33097 } }
I20260806 16:07:20.748759 22141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:07:20.748819 22141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215994295e894902a6f27d9673442894, State: Initialized, Role: FOLLOWER
I20260806 16:07:20.749262 22141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215994295e894902a6f27d9673442894" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33097 } }
I20260806 16:07:20.749390 22141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:07:20.749435 22141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:07:20.749516 22141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:07:20.751991 22141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215994295e894902a6f27d9673442894" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33097 } }
I20260806 16:07:20.752549 22141 leader_election.cc:302] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 215994295e894902a6f27d9673442894; no voters:
I20260806 16:07:20.753028 22141 leader_election.cc:288] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:07:20.753250 22149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:07:20.753619 22149 raft_consensus.cc:686] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [term 1 LEADER]: Becoming Leader. State: Replica: 215994295e894902a6f27d9673442894, State: Running, Role: LEADER
I20260806 16:07:20.754102 22149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215994295e894902a6f27d9673442894" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33097 } }
I20260806 16:07:20.755081 22141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:07:20.756635 22153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "215994295e894902a6f27d9673442894" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215994295e894902a6f27d9673442894" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33097 } } }
I20260806 16:07:20.756847 22153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [sys.catalog]: This master's current role is: LEADER
I20260806 16:07:20.756635 22154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 215994295e894902a6f27d9673442894. Latest consensus state: current_term: 1 leader_uuid: "215994295e894902a6f27d9673442894" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "215994295e894902a6f27d9673442894" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 33097 } } }
I20260806 16:07:20.757066 22154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894 [sys.catalog]: This master's current role is: LEADER
W20260806 16:07:20.761969 22173 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:07:20.762238 22173 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:07:20.762362 22159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:07:20.763441 22159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:07:20.772327 22159 catalog_manager.cc:1347] Generated new cluster ID: 7c90573ebb094150ae9e3edd5d18bef5
I20260806 16:07:20.772451 22159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:07:20.794502 22159 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:07:20.795212 22159 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:07:20.813817 22159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 215994295e894902a6f27d9673442894: Generated new TSK 0
I20260806 16:07:20.815400 22159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:07:20.826151 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:33097
--builtin_ntp_servers=127.8.49.20:43467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 16:07:20.970614 22178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:20.971068 22178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:20.971156 22178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:20.975700 22178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:20.975801 22178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:20.975901 22178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:07:20.982437 22178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:33097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:20.984310 22178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:20.986223 22178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:21.000523 22190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:21.000586 22186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:21.000586 22188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:21.001315 22178 server_base.cc:1034] running on GCE node
I20260806 16:07:21.002060 22178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:21.003409 22178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:21.004825 22178 hybrid_clock.cc:648] HybridClock initialized: now 1786032441004745 us; error 97 us; skew 500 ppm
I20260806 16:07:21.008677 22178 webserver.cc:460] Webserver started at http://127.8.49.1:42941/ using document root <none> and password file <none>
I20260806 16:07:21.009752 22178 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:21.010367 22178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:21.010757 22178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:21.014202 22178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "3fe0eeeb9b1d4fe69e83e36ec1f85a65"
format_stamp: "Formatted at 2026-08-06 16:07:21 on dist-test-slave-7z32"
server_key: "050e1ae258f27a081b5f88131716f20a"
server_key_iv: "11e8734df5a8750c152018b0ee3394ba"
server_key_version: "encryptionkey@0"
I20260806 16:07:21.015242 22178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "3fe0eeeb9b1d4fe69e83e36ec1f85a65"
format_stamp: "Formatted at 2026-08-06 16:07:21 on dist-test-slave-7z32"
server_key: "050e1ae258f27a081b5f88131716f20a"
server_key_iv: "11e8734df5a8750c152018b0ee3394ba"
server_key_version: "encryptionkey@0"
I20260806 16:07:21.022001 22178 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:07:21.027501 22197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:21.029304 22178 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:07:21.029495 22178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "3fe0eeeb9b1d4fe69e83e36ec1f85a65"
format_stamp: "Formatted at 2026-08-06 16:07:21 on dist-test-slave-7z32"
server_key: "050e1ae258f27a081b5f88131716f20a"
server_key_iv: "11e8734df5a8750c152018b0ee3394ba"
server_key_version: "encryptionkey@0"
I20260806 16:07:21.029665 22178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:21.036600 22178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:21.037441 22178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:21.037622 22178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:21.038221 22178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:21.039440 22178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:21.039492 22178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:21.039535 22178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:21.039551 22178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:21.050933 22178 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:42425
I20260806 16:07:21.051083 22371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:42425 every 8 connection(s)
I20260806 16:07:21.052189 22178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:07:21.057502 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 22178
I20260806 16:07:21.057695 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:07:21.058211 8388 external_mini_cluster.cc:1428] Setting key 2f2430c872d850223175a2393d3cd820
I20260806 16:07:21.061518 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:33097
--builtin_ntp_servers=127.8.49.20:43467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 16:07:21.063196 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:21.055597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:35617 (local address 127.8.49.62:33097)
0806 16:07:21.055823 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:21.055830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:21.057958 (+ 2128us) server_negotiation.cc:407] Connection header received
0806 16:07:21.060163 (+ 2205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:21.060169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:21.060221 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:21.060364 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:21.062491 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:21.062495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:21.062497 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:07:21.062513 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:21.062544 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:07:21.062595 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:21.062712 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:21.062988 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 16:07:21.063048 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 16:07:21.065042 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:21.056281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:33097 (local address 127.8.49.1:35617)
0806 16:07:21.057635 (+ 1354us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:21.057755 (+ 120us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:21.059776 (+ 2021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:21.060729 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:21.060743 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:21.060918 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:21.061982 (+ 1064us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:07:21.061986 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:21.062071 (+ 85us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:07:21.062227 (+ 156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:21.062729 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:21.062743 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:21.062754 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 16:07:21.063065 (+ 311us) client_negotiation.cc:240] Negotiation successful
0806 16:07:21.063649 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":175,"threads_started":1}
I20260806 16:07:21.067201 22372 heartbeater.cc:347] Connected to a master server at 127.8.49.62:33097
I20260806 16:07:21.067629 22372 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:21.068508 22372 heartbeater.cc:511] Master 127.8.49.62:33097 requested a full tablet report, sending...
I20260806 16:07:21.071704 22098 ts_manager.cc:194] Registered new tserver with Master: 3fe0eeeb9b1d4fe69e83e36ec1f85a65 (127.8.49.1:42425)
W20260806 16:07:21.231024 22376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:21.231545 22376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:21.231653 22376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:21.237361 22376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:21.237534 22376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:21.237658 22376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:07:21.244840 22376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:33097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:21.246609 22376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:21.248685 22376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:21.262852 22384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:21.262782 22383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:21.263062 22386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:21.263715 22376 server_base.cc:1034] running on GCE node
I20260806 16:07:21.265014 22376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:21.266009 22376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:21.267395 22376 hybrid_clock.cc:648] HybridClock initialized: now 1786032441267359 us; error 57 us; skew 500 ppm
I20260806 16:07:21.271247 22376 webserver.cc:460] Webserver started at http://127.8.49.2:42897/ using document root <none> and password file <none>
I20260806 16:07:21.272207 22376 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:21.272302 22376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:21.272625 22376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:21.276573 22376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "a044d1614c8c48e8a81277683fc37413"
format_stamp: "Formatted at 2026-08-06 16:07:21 on dist-test-slave-7z32"
server_key: "0a8723a548d939f94c86601161d221ea"
server_key_iv: "ba6939991a1b844cc24c5d338293d306"
server_key_version: "encryptionkey@0"
I20260806 16:07:21.277562 22376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "a044d1614c8c48e8a81277683fc37413"
format_stamp: "Formatted at 2026-08-06 16:07:21 on dist-test-slave-7z32"
server_key: "0a8723a548d939f94c86601161d221ea"
server_key_iv: "ba6939991a1b844cc24c5d338293d306"
server_key_version: "encryptionkey@0"
I20260806 16:07:21.284607 22376 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 16:07:21.288934 22394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:21.291126 22376 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:07:21.291401 22376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "a044d1614c8c48e8a81277683fc37413"
format_stamp: "Formatted at 2026-08-06 16:07:21 on dist-test-slave-7z32"
server_key: "0a8723a548d939f94c86601161d221ea"
server_key_iv: "ba6939991a1b844cc24c5d338293d306"
server_key_version: "encryptionkey@0"
I20260806 16:07:21.291617 22376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:21.298126 22376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:21.299157 22376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:21.299467 22376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:21.300673 22376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:21.302764 22376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:21.302953 22376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 16:07:21.303056 22376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:21.303098 22376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:21.320919 22376 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:44367
I20260806 16:07:21.320942 22558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:44367 every 8 connection(s)
I20260806 16:07:21.322351 22376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:07:21.326206 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 22376
I20260806 16:07:21.326419 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:07:21.327112 8388 external_mini_cluster.cc:1428] Setting key 20ad098f62f313d366ac4a3b4bf80bc0
I20260806 16:07:21.330554 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:33097
--builtin_ntp_servers=127.8.49.20:43467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 16:07:21.335130 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:21.325980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:44311 (local address 127.8.49.62:33097)
0806 16:07:21.326189 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:21.326193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:21.329137 (+ 2944us) server_negotiation.cc:407] Connection header received
0806 16:07:21.331665 (+ 2528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:21.331670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:21.331727 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:21.331849 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:21.334227 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:21.334235 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:21.334239 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:07:21.334265 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:21.334305 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:07:21.334319 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:21.334512 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:21.334779 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 16:07:21.334901 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 16:07:21.336987 22562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:21.327352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:33097 (local address 127.8.49.2:44311)
0806 16:07:21.328866 (+ 1514us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:21.328955 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:21.331195 (+ 2240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:21.332318 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:21.332337 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:21.332558 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:21.333570 (+ 1012us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:07:21.333575 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:21.333687 (+ 112us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:07:21.333872 (+ 185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:21.334541 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:21.334558 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:21.334568 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 16:07:21.334882 (+ 314us) client_negotiation.cc:240] Negotiation successful
0806 16:07:21.335290 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":268,"threads_started":1}
I20260806 16:07:21.339535 22559 heartbeater.cc:347] Connected to a master server at 127.8.49.62:33097
I20260806 16:07:21.339990 22559 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:21.340873 22559 heartbeater.cc:511] Master 127.8.49.62:33097 requested a full tablet report, sending...
I20260806 16:07:21.343300 22098 ts_manager.cc:194] Registered new tserver with Master: a044d1614c8c48e8a81277683fc37413 (127.8.49.2:44367)
W20260806 16:07:21.490455 22563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:21.490803 22563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:21.490876 22563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:21.496189 22563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:21.496342 22563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:21.496443 22563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:07:21.501962 22563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:43467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:33097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:21.503396 22563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:21.504940 22563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:21.517448 22572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:21.517453 22569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:21.517453 22570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:21.517961 22563 server_base.cc:1034] running on GCE node
I20260806 16:07:21.518515 22563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:21.519387 22563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:21.522802 22563 hybrid_clock.cc:648] HybridClock initialized: now 1786032441522735 us; error 79 us; skew 500 ppm
I20260806 16:07:21.525736 22563 webserver.cc:460] Webserver started at http://127.8.49.3:40827/ using document root <none> and password file <none>
I20260806 16:07:21.526551 22563 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:21.526634 22563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:21.527037 22563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:21.529589 22563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "423ec507e724464a970a89bab9b0219a"
format_stamp: "Formatted at 2026-08-06 16:07:21 on dist-test-slave-7z32"
server_key: "30eb22cace7557b7b93210fdf8ed6aa2"
server_key_iv: "595fff36115adc3027d9006ad2f42b1c"
server_key_version: "encryptionkey@0"
I20260806 16:07:21.530298 22563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "423ec507e724464a970a89bab9b0219a"
format_stamp: "Formatted at 2026-08-06 16:07:21 on dist-test-slave-7z32"
server_key: "30eb22cace7557b7b93210fdf8ed6aa2"
server_key_iv: "595fff36115adc3027d9006ad2f42b1c"
server_key_version: "encryptionkey@0"
I20260806 16:07:21.535699 22563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:07:21.539724 22579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:21.541801 22563 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:07:21.541968 22563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "423ec507e724464a970a89bab9b0219a"
format_stamp: "Formatted at 2026-08-06 16:07:21 on dist-test-slave-7z32"
server_key: "30eb22cace7557b7b93210fdf8ed6aa2"
server_key_iv: "595fff36115adc3027d9006ad2f42b1c"
server_key_version: "encryptionkey@0"
I20260806 16:07:21.542196 22563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:21.548014 22563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:21.548808 22563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:21.549049 22563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:21.549772 22563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:21.551054 22563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:21.551115 22563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:21.551160 22563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:21.551177 22563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:21.564385 22563 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:45745
I20260806 16:07:21.564422 22752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:45745 every 8 connection(s)
I20260806 16:07:21.566012 22563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:07:21.574815 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 22563
I20260806 16:07:21.575065 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:07:21.575538 8388 external_mini_cluster.cc:1428] Setting key 1ac108e0e45f7d9d93183ad7d2c74088
I20260806 16:07:21.575922 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:21.569101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:40649 (local address 127.8.49.62:33097)
0806 16:07:21.569280 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:21.569284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:21.571188 (+ 1904us) server_negotiation.cc:407] Connection header received
0806 16:07:21.573324 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:21.573328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:21.573392 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:21.573502 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:21.574998 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:21.575002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:21.575006 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:07:21.575024 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:21.575080 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:07:21.575094 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:21.575205 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:21.575704 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 16:07:21.575770 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:07:21.578159 22756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:21.569702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:33097 (local address 127.8.49.3:40649)
0806 16:07:21.570764 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:21.570894 (+ 130us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:21.572874 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:21.573740 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:21.573749 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:21.573866 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:21.574612 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:07:21.574614 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:21.574668 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:07:21.574755 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:21.575286 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:21.575311 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:21.575327 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 16:07:21.575703 (+ 376us) client_negotiation.cc:240] Negotiation successful
0806 16:07:21.576194 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":190,"threads_started":1}
I20260806 16:07:21.580242 22753 heartbeater.cc:347] Connected to a master server at 127.8.49.62:33097
I20260806 16:07:21.580703 22753 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:21.581455 22753 heartbeater.cc:511] Master 127.8.49.62:33097 requested a full tablet report, sending...
I20260806 16:07:21.583853 22098 ts_manager.cc:194] Registered new tserver with Master: 423ec507e724464a970a89bab9b0219a (127.8.49.3:45745)
I20260806 16:07:21.593266 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 16:07:21.598507 22147 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:07:21.597183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41506 (local address 127.8.49.62:33097)
0806 16:07:21.597384 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:21.597388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:21.597581 (+ 193us) server_negotiation.cc:407] Connection header received
0806 16:07:21.597837 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:21.597839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:21.597885 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:21.597974 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:21.598323 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41506: BlockingRecv error: recv got EOF from 127.0.0.1:41506 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 16:07:21.598793 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 22178
I20260806 16:07:21.612285 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 22376
I20260806 16:07:21.631604 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 22563
I20260806 16:07:21.657914 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 22055
2026-08-06T16:07:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (1249 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:07:21 dist-test-slave-7z32 krb5kdc[22768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:07:21 dist-test-slave-7z32 krb5kdc[22768](info): set up 2 sockets
Aug 06 16:07:21 dist-test-slave-7z32 krb5kdc[22768](info): commencing operation
krb5kdc: starting...
W20260806 16:07:23.069479 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.369s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:07:23 dist-test-slave-7z32 krb5kdc[22768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:07:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:07:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:07:23.251478 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:34047
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:34047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:07:23.406361 22784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:23.406725 22784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:23.406772 22784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:23.412179 22784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:07:23.412276 22784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:23.412300 22784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:23.412320 22784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:07:23.412343 22784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:07:23.419126 22784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:34047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:34047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:23.420481 22784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:23.421744 22784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:23.433665 22791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:23.433808 22784 server_base.cc:1034] running on GCE node
W20260806 16:07:23.433665 22789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:23.433693 22793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:23.435446 22784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:23.437129 22784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:23.438632 22784 hybrid_clock.cc:648] HybridClock initialized: now 1786032443438571 us; error 74 us; skew 500 ppm
Aug 06 16:07:23 dist-test-slave-7z32 krb5kdc[22768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032443, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:23.443653 22784 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:07:23.445083 22784 webserver.cc:460] Webserver started at http://127.8.49.62:42179/ using document root <none> and password file <none>
I20260806 16:07:23.445911 22784 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:23.446025 22784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:23.446309 22784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:23.449589 22784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "325bd16bafff4e08912a58ac582cd875"
format_stamp: "Formatted at 2026-08-06 16:07:23 on dist-test-slave-7z32"
server_key: "b5e900ed699c2eb5a1abaff9610fb332"
server_key_iv: "144b829e09d25bbf9ecae5c588a3b6d4"
server_key_version: "encryptionkey@0"
I20260806 16:07:23.450541 22784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "325bd16bafff4e08912a58ac582cd875"
format_stamp: "Formatted at 2026-08-06 16:07:23 on dist-test-slave-7z32"
server_key: "b5e900ed699c2eb5a1abaff9610fb332"
server_key_iv: "144b829e09d25bbf9ecae5c588a3b6d4"
server_key_version: "encryptionkey@0"
I20260806 16:07:23.456560 22784 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.002s
I20260806 16:07:23.460954 22803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:23.463658 22784 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:07:23.463969 22784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "325bd16bafff4e08912a58ac582cd875"
format_stamp: "Formatted at 2026-08-06 16:07:23 on dist-test-slave-7z32"
server_key: "b5e900ed699c2eb5a1abaff9610fb332"
server_key_iv: "144b829e09d25bbf9ecae5c588a3b6d4"
server_key_version: "encryptionkey@0"
I20260806 16:07:23.464188 22784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:23.504036 22784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:23.509310 22784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:23.509632 22784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:23.520563 22784 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:34047
I20260806 16:07:23.521116 22881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:34047 every 8 connection(s)
I20260806 16:07:23.522058 22784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:07:23.524610 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 22784
I20260806 16:07:23.524773 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:07:23.525148 8388 external_mini_cluster.cc:1428] Setting key 9fc32ac743b6049f8b8185d34b259918
I20260806 16:07:23.526440 22882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:07:23.535689 22882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875: Bootstrap starting.
Aug 06 16:07:23 dist-test-slave-7z32 krb5kdc[22768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:07:23.539269 22882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875: Neither blocks nor log segments found. Creating new log.
I20260806 16:07:23.540261 22882 log.cc:824] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875: Log is configured to *not* fsync() on all Append() calls
I20260806 16:07:23.542660 22882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875: No bootstrap required, opened a new log
I20260806 16:07:23.545826 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:23.527333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.8.49.62:34047)
0806 16:07:23.527962 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:23.527986 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:23.528041 (+ 55us) server_negotiation.cc:407] Connection header received
0806 16:07:23.529008 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:23.529031 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:23.529521 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:23.529943 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:23.531224 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:23.532773 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:23.534258 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:23.535157 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:23.535354 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:23.539202 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:23.539211 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:23.539237 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:07:23.539293 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:23.541744 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:23.542434 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:23.542442 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:23.542450 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:23.542602 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:23.543016 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:23.543020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:23.543021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:23.543626 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:23.543899 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:23.544285 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 16:07:23.544721 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":202,"threads_started":1}
I20260806 16:07:23.547082 22882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd16bafff4e08912a58ac582cd875" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34047 } }
I20260806 16:07:23.547406 22882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:07:23.547469 22882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325bd16bafff4e08912a58ac582cd875, State: Initialized, Role: FOLLOWER
I20260806 16:07:23.548012 22882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd16bafff4e08912a58ac582cd875" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34047 } }
I20260806 16:07:23.548187 22882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:07:23.548287 22882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:07:23.548429 22882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:07:23.549685 22882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd16bafff4e08912a58ac582cd875" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34047 } }
I20260806 16:07:23.550117 22882 leader_election.cc:302] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 325bd16bafff4e08912a58ac582cd875; no voters:
I20260806 16:07:23.550551 22882 leader_election.cc:288] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:07:23.550870 22890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:07:23.551393 22890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [term 1 LEADER]: Becoming Leader. State: Replica: 325bd16bafff4e08912a58ac582cd875, State: Running, Role: LEADER
I20260806 16:07:23.551867 22890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd16bafff4e08912a58ac582cd875" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34047 } }
I20260806 16:07:23.552512 22882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:07:23.554234 22891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "325bd16bafff4e08912a58ac582cd875" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd16bafff4e08912a58ac582cd875" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34047 } } }
I20260806 16:07:23.554436 22891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [sys.catalog]: This master's current role is: LEADER
I20260806 16:07:23.554805 22895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 325bd16bafff4e08912a58ac582cd875. Latest consensus state: current_term: 1 leader_uuid: "325bd16bafff4e08912a58ac582cd875" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd16bafff4e08912a58ac582cd875" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34047 } } }
I20260806 16:07:23.554960 22895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875 [sys.catalog]: This master's current role is: LEADER
W20260806 16:07:23.558432 22912 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:07:23.558655 22912 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:07:23.558908 22905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:07:23.560820 22905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:07:23.570677 22905 catalog_manager.cc:1347] Generated new cluster ID: d6aa4c8b61844c10b55db11860b48329
I20260806 16:07:23.570858 22905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:07:23.599820 22905 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:07:23.601090 22905 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:07:23.615036 22905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875: Generated new TSK 0
I20260806 16:07:23.616113 22905 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:07:23.667701 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34047
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:07:23.841064 22921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:23.841495 22921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:23.841553 22921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:23.846457 22921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:23.846570 22921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:23.846690 22921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:07:23.853871 22921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:23.855724 22921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:23.857219 22921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:23.871021 22927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:23.871178 22928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:23.871125 22921 server_base.cc:1034] running on GCE node
W20260806 16:07:23.871025 22930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:23.872186 22921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:23.873267 22921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:23.874522 22921 hybrid_clock.cc:648] HybridClock initialized: now 1786032443874473 us; error 73 us; skew 500 ppm
Aug 06 16:07:23 dist-test-slave-7z32 krb5kdc[22768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032443, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:23.886596 22921 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:07:23.888486 22921 webserver.cc:460] Webserver started at http://127.8.49.1:46489/ using document root <none> and password file <none>
I20260806 16:07:23.889454 22921 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:23.889592 22921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:23.889945 22921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:23.893224 22921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "4ddf2322395047c88574377a755e0cec"
format_stamp: "Formatted at 2026-08-06 16:07:23 on dist-test-slave-7z32"
server_key: "4047418f303375b7fa434e444f549f14"
server_key_iv: "fb9b957ccddcfa776857f8251b46f39e"
server_key_version: "encryptionkey@0"
I20260806 16:07:23.894546 22921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "4ddf2322395047c88574377a755e0cec"
format_stamp: "Formatted at 2026-08-06 16:07:23 on dist-test-slave-7z32"
server_key: "4047418f303375b7fa434e444f549f14"
server_key_iv: "fb9b957ccddcfa776857f8251b46f39e"
server_key_version: "encryptionkey@0"
I20260806 16:07:23.900736 22921 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260806 16:07:23.904479 22940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:23.906387 22921 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:07:23.906550 22921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "4ddf2322395047c88574377a755e0cec"
format_stamp: "Formatted at 2026-08-06 16:07:23 on dist-test-slave-7z32"
server_key: "4047418f303375b7fa434e444f549f14"
server_key_iv: "fb9b957ccddcfa776857f8251b46f39e"
server_key_version: "encryptionkey@0"
I20260806 16:07:23.906754 22921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:23.931262 22921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:23.935745 22921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:23.935983 22921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:23.936729 22921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:23.937850 22921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:23.937912 22921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:23.937960 22921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:23.938033 22921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:23.950192 22921 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:36469
I20260806 16:07:23.950294 23085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:36469 every 8 connection(s)
I20260806 16:07:23.951510 22921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:07:23.952724 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 22921
I20260806 16:07:23.952926 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:07:23.953264 8388 external_mini_cluster.cc:1428] Setting key 6a6d6ba51a195f9dd069646e657eb53e
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
Aug 06 16:07:23 dist-test-slave-7z32 krb5kdc[22768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032443, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:07:23.975111 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:23.954949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:54215 (local address 127.8.49.62:34047)
0806 16:07:23.955134 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:23.955140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:23.956954 (+ 1814us) server_negotiation.cc:407] Connection header received
0806 16:07:23.959639 (+ 2685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:23.959653 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:23.959746 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:23.959842 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:23.961605 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:23.962219 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:23.964045 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:23.964344 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:23.964477 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:23.968507 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:23.968511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:23.968514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:07:23.968552 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:23.971379 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:23.972690 (+ 1311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:23.972695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:23.972697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:23.972857 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:23.973419 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:23.973422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:23.973425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:23.973714 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:23.973856 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:23.974634 (+ 778us) server_negotiation.cc:299] Negotiation successful
0806 16:07:23.974886 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 16:07:23.976401 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:23.955656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:34047 (local address 127.8.49.1:54215)
0806 16:07:23.956441 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:23.956712 (+ 271us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:23.959256 (+ 2544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:23.960137 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:23.960155 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:23.960699 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:23.961390 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:23.961407 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:23.962383 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:23.962391 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:23.963885 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:23.963899 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:23.964567 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:23.964572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:23.964801 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:23.965867 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:23.965892 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:23.968146 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:23.971609 (+ 3463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:23.971618 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:23.971641 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:23.972353 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:23.973003 (+ 650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:23.973007 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:23.973009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:23.973256 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:23.973858 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:23.973873 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:23.974299 (+ 426us) client_negotiation.cc:769] Sending connection context
0806 16:07:23.974572 (+ 273us) client_negotiation.cc:240] Negotiation successful
0806 16:07:23.975137 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":181,"threads_started":1}
I20260806 16:07:23.978890 23089 heartbeater.cc:347] Connected to a master server at 127.8.49.62:34047
I20260806 16:07:23.979363 23089 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:23.980242 23089 heartbeater.cc:511] Master 127.8.49.62:34047 requested a full tablet report, sending...
I20260806 16:07:23.982946 22833 ts_manager.cc:194] Registered new tserver with Master: 4ddf2322395047c88574377a755e0cec (127.8.49.1:36469)
I20260806 16:07:23.984356 22833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:54215
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:07:23.995584 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34047
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:07:24.139048 23096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:24.139425 23096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:24.139477 23096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:24.143981 23096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:24.144084 23096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:24.144152 23096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:07:24.149947 23096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:24.151214 23096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:24.152335 23096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:24.164754 23102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:24.164772 23103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:24.164846 23096 server_base.cc:1034] running on GCE node
W20260806 16:07:24.164754 23105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:24.165838 23096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:24.166666 23096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:24.167893 23096 hybrid_clock.cc:648] HybridClock initialized: now 1786032444167852 us; error 63 us; skew 500 ppm
Aug 06 16:07:24 dist-test-slave-7z32 krb5kdc[22768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032444, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:24.172478 23096 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:07:24.173841 23096 webserver.cc:460] Webserver started at http://127.8.49.2:41259/ using document root <none> and password file <none>
I20260806 16:07:24.174566 23096 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:24.174655 23096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:24.174927 23096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:24.177325 23096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "e026575bc6a44d48b25303e1b946f322"
format_stamp: "Formatted at 2026-08-06 16:07:24 on dist-test-slave-7z32"
server_key: "428b715f6cd0aff10f35be37d708f25e"
server_key_iv: "c575a36b5d209112dd12c33b8688f035"
server_key_version: "encryptionkey@0"
I20260806 16:07:24.177934 23096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "e026575bc6a44d48b25303e1b946f322"
format_stamp: "Formatted at 2026-08-06 16:07:24 on dist-test-slave-7z32"
server_key: "428b715f6cd0aff10f35be37d708f25e"
server_key_iv: "c575a36b5d209112dd12c33b8688f035"
server_key_version: "encryptionkey@0"
I20260806 16:07:24.183125 23096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 16:07:24.186654 23118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:24.188776 23096 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:07:24.188946 23096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "e026575bc6a44d48b25303e1b946f322"
format_stamp: "Formatted at 2026-08-06 16:07:24 on dist-test-slave-7z32"
server_key: "428b715f6cd0aff10f35be37d708f25e"
server_key_iv: "c575a36b5d209112dd12c33b8688f035"
server_key_version: "encryptionkey@0"
I20260806 16:07:24.189076 23096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:24.205969 23096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:24.210287 23096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:24.210491 23096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:24.211316 23096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:24.212524 23096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:24.212594 23096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:24.212639 23096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:24.212656 23096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:24.225931 23096 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:43665
I20260806 16:07:24.225977 23291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:43665 every 8 connection(s)
I20260806 16:07:24.227394 23096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:07:24.237159 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 23096
I20260806 16:07:24.237353 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:07:24.237780 8388 external_mini_cluster.cc:1428] Setting key 68a15b7546fa85db251f941dfd22d874
Aug 06 16:07:24 dist-test-slave-7z32 krb5kdc[22768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032444, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:07:24.248467 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.230589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:45231 (local address 127.8.49.62:34047)
0806 16:07:24.230871 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:24.230879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:24.232322 (+ 1443us) server_negotiation.cc:407] Connection header received
0806 16:07:24.234189 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:24.234192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:24.234288 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:24.234409 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:24.236340 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.236976 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.239158 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.239490 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.239615 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.244211 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:24.244214 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:24.244217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:07:24.244248 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:24.245892 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:24.246668 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:24.246672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:24.246673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:24.246752 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:24.247182 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:24.247185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:24.247187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:24.247406 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:24.247522 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:24.248131 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 16:07:24.248281 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:07:24.249446 23295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.231129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:34047 (local address 127.8.49.2:45231)
0806 16:07:24.231880 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:24.232069 (+ 189us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:24.233858 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:24.234737 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:24.234748 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:24.235374 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:24.236092 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.236112 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.237269 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.237276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.238975 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.238993 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.239730 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.239735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.240023 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.240931 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:24.240959 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:24.243932 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:24.246066 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:24.246074 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:24.246098 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:24.246540 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:24.246873 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:24.246878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:24.246881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:24.247059 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:24.247522 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:24.247531 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:24.247854 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 16:07:24.248066 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 16:07:24.248484 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":187,"threads_started":1}
I20260806 16:07:24.251803 23292 heartbeater.cc:347] Connected to a master server at 127.8.49.62:34047
I20260806 16:07:24.252236 23292 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:24.252981 23292 heartbeater.cc:511] Master 127.8.49.62:34047 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:07:24.255127 22833 ts_manager.cc:194] Registered new tserver with Master: e026575bc6a44d48b25303e1b946f322 (127.8.49.2:43665)
I20260806 16:07:24.255945 22833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:45231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:07:24.286731 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34047
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:07:24.443714 23301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:07:24.444087 23301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:07:24.444165 23301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:07:24.448534 23301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:07:24.448647 23301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:07:24.448745 23301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:07:24.455143 23301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:07:24.456481 23301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:07:24.457652 23301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:07:24.469811 23310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:07:24.469831 23307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:24.469898 23301 server_base.cc:1034] running on GCE node
W20260806 16:07:24.469815 23308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:07:24.470885 23301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:07:24.471781 23301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:07:24.473141 23301 hybrid_clock.cc:648] HybridClock initialized: now 1786032444473120 us; error 50 us; skew 500 ppm
Aug 06 16:07:24 dist-test-slave-7z32 krb5kdc[22768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032444, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:07:24.476940 23301 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:07:24.478215 23301 webserver.cc:460] Webserver started at http://127.8.49.3:42665/ using document root <none> and password file <none>
I20260806 16:07:24.479116 23301 fs_manager.cc:359] Metadata directory not provided
I20260806 16:07:24.479203 23301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:07:24.479489 23301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:07:24.482093 23301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf"
format_stamp: "Formatted at 2026-08-06 16:07:24 on dist-test-slave-7z32"
server_key: "e3b6ddbba13d6034670d2e12928dd23c"
server_key_iv: "112a7cc3aa45564818037dd447358e04"
server_key_version: "encryptionkey@0"
I20260806 16:07:24.482774 23301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf"
format_stamp: "Formatted at 2026-08-06 16:07:24 on dist-test-slave-7z32"
server_key: "e3b6ddbba13d6034670d2e12928dd23c"
server_key_iv: "112a7cc3aa45564818037dd447358e04"
server_key_version: "encryptionkey@0"
I20260806 16:07:24.488125 23301 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:07:24.491605 23321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:24.493590 23301 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:07:24.493744 23301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf"
format_stamp: "Formatted at 2026-08-06 16:07:24 on dist-test-slave-7z32"
server_key: "e3b6ddbba13d6034670d2e12928dd23c"
server_key_iv: "112a7cc3aa45564818037dd447358e04"
server_key_version: "encryptionkey@0"
I20260806 16:07:24.493933 23301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:07:24.508563 23301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:07:24.512552 23301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:07:24.512748 23301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:07:24.513479 23301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:07:24.514549 23301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:07:24.514611 23301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:24.514662 23301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:07:24.514679 23301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:07:24.529366 23301 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:46523
I20260806 16:07:24.529464 23501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:46523 every 8 connection(s)
I20260806 16:07:24.530706 23301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:07:24.539506 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 23301
I20260806 16:07:24.539671 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:07:24.540095 8388 external_mini_cluster.cc:1428] Setting key c99cf7918b174a1e4d270438b8a7f816
Aug 06 16:07:24 dist-test-slave-7z32 krb5kdc[22768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032444, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:07:24.552062 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.533159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:45935 (local address 127.8.49.62:34047)
0806 16:07:24.533404 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:24.533410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:24.534918 (+ 1508us) server_negotiation.cc:407] Connection header received
0806 16:07:24.536953 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:24.536957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:24.537040 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:24.537160 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:24.539284 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.540067 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.541887 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.542192 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.542316 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.546388 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:24.546393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:24.546396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:07:24.546460 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:24.548656 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:24.549598 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:24.549603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:24.549604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:24.549728 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:24.550212 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:24.550215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:24.550216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:24.550446 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:24.550728 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:24.551638 (+ 910us) server_negotiation.cc:299] Negotiation successful
0806 16:07:24.551846 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 16:07:24.553232 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.533583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:34047 (local address 127.8.49.3:45935)
0806 16:07:24.534447 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:24.534671 (+ 224us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:24.536587 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:24.537476 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:24.537486 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:24.538086 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:07:24.538956 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.538992 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.540350 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.540355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.541712 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.541731 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.542384 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.542389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.542596 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.543543 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:07:24.543564 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:07:24.545990 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:07:24.548864 (+ 2874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:24.548873 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:24.548892 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:24.549301 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:24.549848 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:07:24.549851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:07:24.549853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:07:24.550059 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:07:24.550797 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:07:24.550805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:07:24.551240 (+ 435us) client_negotiation.cc:769] Sending connection context
0806 16:07:24.551449 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 16:07:24.551875 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":173,"threads_started":1}
I20260806 16:07:24.555347 23502 heartbeater.cc:347] Connected to a master server at 127.8.49.62:34047
I20260806 16:07:24.555696 23502 heartbeater.cc:464] Registering TS with master...
I20260806 16:07:24.556422 23502 heartbeater.cc:511] Master 127.8.49.62:34047 requested a full tablet report, sending...
I20260806 16:07:24.557816 22833 ts_manager.cc:194] Registered new tserver with Master: e2a62dd7d9b940cebe3d52a7e2bc68cf (127.8.49.3:46523)
I20260806 16:07:24.558636 22833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:45935
I20260806 16:07:24.570811 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:07:24.586642 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.574913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60520 (local address 127.8.49.62:34047)
0806 16:07:24.575172 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:24.575179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:24.575416 (+ 237us) server_negotiation.cc:407] Connection header received
0806 16:07:24.575644 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:24.575648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:24.575721 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:24.575825 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:07:24.576937 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.577581 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.578607 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.579008 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.579144 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.581740 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:07:24.581745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:07:24.581747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:07:24.581793 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:07:24.584322 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:24.584988 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:24.584992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:24.584994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:24.585104 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:07:24.585434 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:07:24.585438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:07:24.585440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:07:24.585782 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:07:24.585956 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:24.586251 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 16:07:24.586414 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 16:07:24.592818 22833 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:07:24.597645 22833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:07:24.621749 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.614624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:36469 (local address 127.0.0.1:52968)
0806 16:07:24.615439 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:24.615456 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:24.615553 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:24.616421 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:24.616424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:24.616438 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:07:24.616610 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.616617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.618116 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.618124 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.619689 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.619701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.620909 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.620914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.621103 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.621169 (+ 66us) client_negotiation.cc:769] Sending connection context
0806 16:07:24.621336 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 16:07:24.621532 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":7,"thread_start_us":148,"threads_started":1}
I20260806 16:07:24.623116 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.613507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:46523 (local address 127.0.0.1:41838)
0806 16:07:24.615163 (+ 1656us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:24.615183 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:24.615322 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:24.616277 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:24.616284 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:24.616309 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:07:24.616608 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.616617 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.618676 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.618679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.620484 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.620504 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.622498 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.622504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.622641 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.622679 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 16:07:24.622758 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 16:07:24.622914 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1565,"mutex_wait_us":248,"thread_start_us":207,"threads_started":1}
I20260806 16:07:24.623778 23525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.613572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35382 (local address 127.8.49.2:43665)
0806 16:07:24.614472 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:24.614488 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:24.615046 (+ 558us) server_negotiation.cc:407] Connection header received
0806 16:07:24.615356 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:24.615366 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:24.615638 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:24.615856 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:07:24.616851 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.618398 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.620770 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.622213 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.622362 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.622705 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:24.623423 (+ 718us) server_negotiation.cc:299] Negotiation successful
0806 16:07:24.623566 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":180,"threads_started":1}
I20260806 16:07:24.624191 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.614815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.8.49.3:46523)
0806 16:07:24.615677 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:24.615693 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:24.615717 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:07:24.615789 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:24.615794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:24.615960 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:24.616785 (+ 825us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:07:24.616978 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.618398 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.620743 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.622260 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.623413 (+ 1153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.623639 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:24.623819 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 16:07:24.623994 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":430,"threads_started":1}
I20260806 16:07:24.624648 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.613291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:43665 (local address 127.0.0.1:35382)
0806 16:07:24.614129 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:24.614235 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:24.614737 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:24.616295 (+ 1558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:24.616298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:24.616326 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:07:24.616599 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.616605 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.618644 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.618652 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.620482 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.620504 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.622800 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.622804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.623022 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.623055 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 16:07:24.623149 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 16:07:24.624451 (+ 1302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":154,"thread_start_us":139,"threads_started":1}
I20260806 16:07:24.624691 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.614855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.8.49.1:36469)
0806 16:07:24.615292 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:24.615300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:24.615658 (+ 358us) server_negotiation.cc:407] Connection header received
0806 16:07:24.615759 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:24.615765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:24.615960 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:24.616242 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:07:24.616802 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.617910 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.619869 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.620723 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.620870 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.621105 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:24.621413 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 16:07:24.622103 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":86,"threads_started":1}
I20260806 16:07:24.629823 23001 tablet_service.cc:1467] Processing CreateTablet for tablet 5909c9142d7747df819ab2fd51210b80 (DEFAULT_TABLE table=test-table [id=c7a5c628a67f4a0aa154a707d6328611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:07:24.631639 23001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5909c9142d7747df819ab2fd51210b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:07:24.634917 23394 tablet_service.cc:1467] Processing CreateTablet for tablet 5909c9142d7747df819ab2fd51210b80 (DEFAULT_TABLE table=test-table [id=c7a5c628a67f4a0aa154a707d6328611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:07:24.637086 23394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5909c9142d7747df819ab2fd51210b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:07:24.636500 23191 tablet_service.cc:1467] Processing CreateTablet for tablet 5909c9142d7747df819ab2fd51210b80 (DEFAULT_TABLE table=test-table [id=c7a5c628a67f4a0aa154a707d6328611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:07:24.638370 23191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5909c9142d7747df819ab2fd51210b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:07:24.645385 23532 tablet_bootstrap.cc:492] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec: Bootstrap starting.
I20260806 16:07:24.648738 23532 tablet_bootstrap.cc:654] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec: Neither blocks nor log segments found. Creating new log.
I20260806 16:07:24.650298 23532 log.cc:824] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec: Log is configured to *not* fsync() on all Append() calls
I20260806 16:07:24.651384 23533 tablet_bootstrap.cc:492] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: Bootstrap starting.
I20260806 16:07:24.652411 23536 tablet_bootstrap.cc:492] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: Bootstrap starting.
I20260806 16:07:24.653370 23532 tablet_bootstrap.cc:492] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec: No bootstrap required, opened a new log
I20260806 16:07:24.653650 23532 ts_tablet_manager.cc:1397] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260806 16:07:24.654845 23533 tablet_bootstrap.cc:654] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: Neither blocks nor log segments found. Creating new log.
I20260806 16:07:24.656082 23536 tablet_bootstrap.cc:654] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: Neither blocks nor log segments found. Creating new log.
I20260806 16:07:24.656813 23533 log.cc:824] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: Log is configured to *not* fsync() on all Append() calls
I20260806 16:07:24.657122 23536 log.cc:824] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: Log is configured to *not* fsync() on all Append() calls
I20260806 16:07:24.660023 23533 tablet_bootstrap.cc:492] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: No bootstrap required, opened a new log
I20260806 16:07:24.659988 23532 raft_consensus.cc:348] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e026575bc6a44d48b25303e1b946f322" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 43665 } } peers { permanent_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46523 } } peers { permanent_uuid: "4ddf2322395047c88574377a755e0cec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 36469 } }
I20260806 16:07:24.660347 23536 tablet_bootstrap.cc:492] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: No bootstrap required, opened a new log
I20260806 16:07:24.660352 23533 ts_tablet_manager.cc:1397] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260806 16:07:24.660342 23532 raft_consensus.cc:374] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:07:24.660419 23532 raft_consensus.cc:729] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ddf2322395047c88574377a755e0cec, State: Initialized, Role: FOLLOWER
I20260806 16:07:24.660566 23536 ts_tablet_manager.cc:1397] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 16:07:24.661032 23532 consensus_queue.cc:260] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e026575bc6a44d48b25303e1b946f322" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 43665 } } peers { permanent_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46523 } } peers { permanent_uuid: "4ddf2322395047c88574377a755e0cec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 36469 } }
I20260806 16:07:24.662292 23532 ts_tablet_manager.cc:1428] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260806 16:07:24.662923 23089 heartbeater.cc:503] Master 127.8.49.62:34047 was elected leader, sending a full tablet report...
I20260806 16:07:24.664837 23536 raft_consensus.cc:348] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e026575bc6a44d48b25303e1b946f322" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 43665 } } peers { permanent_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46523 } } peers { permanent_uuid: "4ddf2322395047c88574377a755e0cec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 36469 } }
I20260806 16:07:24.665213 23536 raft_consensus.cc:374] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:07:24.665282 23536 raft_consensus.cc:729] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e026575bc6a44d48b25303e1b946f322, State: Initialized, Role: FOLLOWER
I20260806 16:07:24.665791 23536 consensus_queue.cc:260] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e026575bc6a44d48b25303e1b946f322" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 43665 } } peers { permanent_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46523 } } peers { permanent_uuid: "4ddf2322395047c88574377a755e0cec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 36469 } }
I20260806 16:07:24.666298 23533 raft_consensus.cc:348] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e026575bc6a44d48b25303e1b946f322" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 43665 } } peers { permanent_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46523 } } peers { permanent_uuid: "4ddf2322395047c88574377a755e0cec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 36469 } }
I20260806 16:07:24.666710 23533 raft_consensus.cc:374] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:07:24.666776 23533 raft_consensus.cc:729] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a62dd7d9b940cebe3d52a7e2bc68cf, State: Initialized, Role: FOLLOWER
I20260806 16:07:24.667047 23292 heartbeater.cc:503] Master 127.8.49.62:34047 was elected leader, sending a full tablet report...
I20260806 16:07:24.667399 23533 consensus_queue.cc:260] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e026575bc6a44d48b25303e1b946f322" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 43665 } } peers { permanent_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46523 } } peers { permanent_uuid: "4ddf2322395047c88574377a755e0cec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 36469 } }
I20260806 16:07:24.667527 23536 ts_tablet_manager.cc:1428] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 16:07:24.668865 23502 heartbeater.cc:503] Master 127.8.49.62:34047 was elected leader, sending a full tablet report...
I20260806 16:07:24.669252 23533 ts_tablet_manager.cc:1428] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: Time spent starting tablet: real 0.009s user 0.006s sys 0.002s
W20260806 16:07:24.704087 23090 tablet.cc:2369] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:07:24.729079 23294 tablet.cc:2369] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:07:24.762681 23541 raft_consensus.cc:482] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:07:24.763017 23541 raft_consensus.cc:504] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e026575bc6a44d48b25303e1b946f322" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 43665 } } peers { permanent_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46523 } } peers { permanent_uuid: "4ddf2322395047c88574377a755e0cec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 36469 } }
I20260806 16:07:24.764302 23541 leader_election.cc:288] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a62dd7d9b940cebe3d52a7e2bc68cf (127.8.49.3:46523), 4ddf2322395047c88574377a755e0cec (127.8.49.1:36469)
I20260806 16:07:24.772295 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.765026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:43993 (local address 127.8.49.1:36469)
0806 16:07:24.765192 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:24.765197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:24.765529 (+ 332us) server_negotiation.cc:407] Connection header received
0806 16:07:24.765720 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:24.765723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:24.765784 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:24.765907 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:07:24.766508 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.767466 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.770124 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.771080 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.771212 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.771306 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:24.771986 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 16:07:24.772155 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 16:07:24.772315 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.764788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:46523 (local address 127.8.49.2:46997)
0806 16:07:24.765475 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:24.765489 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:24.765591 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:24.765969 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:24.765973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:24.766028 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:07:24.766308 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.766319 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.767744 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.767750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.769882 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.769903 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.771284 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.771289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.771665 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.771757 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 16:07:24.771881 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 16:07:24.772023 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":698,"spinlock_wait_cycles":47488,"thread_start_us":77,"threads_started":1}
I20260806 16:07:24.772364 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.764766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:36469 (local address 127.8.49.2:43993)
0806 16:07:24.765392 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:24.765413 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:07:24.765591 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:07:24.765969 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:07:24.765973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:07:24.766029 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:07:24.766330 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.766336 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.767744 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.767750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.769882 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:07:24.769903 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.771285 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.771289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:07:24.771662 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.771757 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 16:07:24.771885 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 16:07:24.772023 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":304,"spinlock_wait_cycles":8192,"thread_start_us":141,"threads_started":1}
I20260806 16:07:24.772289 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.765091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:46997 (local address 127.8.49.3:46523)
0806 16:07:24.765227 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:24.765234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:24.765605 (+ 371us) server_negotiation.cc:407] Connection header received
0806 16:07:24.765716 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:24.765719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:24.765784 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:24.765907 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:07:24.766508 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.767519 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.770124 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.771044 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.771212 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.771305 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:24.771986 (+ 681us) server_negotiation.cc:299] Negotiation successful
0806 16:07:24.772109 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:07:24.773397 23421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909c9142d7747df819ab2fd51210b80" candidate_uuid: "e026575bc6a44d48b25303e1b946f322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" is_pre_election: true
I20260806 16:07:24.773397 23027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909c9142d7747df819ab2fd51210b80" candidate_uuid: "e026575bc6a44d48b25303e1b946f322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ddf2322395047c88574377a755e0cec" is_pre_election: true
I20260806 16:07:24.773907 23027 raft_consensus.cc:2454] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e026575bc6a44d48b25303e1b946f322 in term 0.
I20260806 16:07:24.773907 23421 raft_consensus.cc:2454] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e026575bc6a44d48b25303e1b946f322 in term 0.
I20260806 16:07:24.774714 23119 leader_election.cc:302] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4ddf2322395047c88574377a755e0cec, e026575bc6a44d48b25303e1b946f322, e2a62dd7d9b940cebe3d52a7e2bc68cf; no voters:
I20260806 16:07:24.775251 23541 raft_consensus.cc:2781] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:07:24.775480 23541 raft_consensus.cc:482] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:07:24.775574 23541 raft_consensus.cc:3037] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:07:24.776962 23541 raft_consensus.cc:504] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e026575bc6a44d48b25303e1b946f322" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 43665 } } peers { permanent_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46523 } } peers { permanent_uuid: "4ddf2322395047c88574377a755e0cec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 36469 } }
I20260806 16:07:24.777438 23541 leader_election.cc:288] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [CANDIDATE]: Term 1 election: Requested vote from peers e2a62dd7d9b940cebe3d52a7e2bc68cf (127.8.49.3:46523), 4ddf2322395047c88574377a755e0cec (127.8.49.1:36469)
I20260806 16:07:24.778112 23027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909c9142d7747df819ab2fd51210b80" candidate_uuid: "e026575bc6a44d48b25303e1b946f322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ddf2322395047c88574377a755e0cec"
I20260806 16:07:24.778111 23421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909c9142d7747df819ab2fd51210b80" candidate_uuid: "e026575bc6a44d48b25303e1b946f322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf"
I20260806 16:07:24.778319 23027 raft_consensus.cc:3037] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:07:24.778388 23421 raft_consensus.cc:3037] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:07:24.780298 23421 raft_consensus.cc:2454] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e026575bc6a44d48b25303e1b946f322 in term 1.
I20260806 16:07:24.780288 23027 raft_consensus.cc:2454] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e026575bc6a44d48b25303e1b946f322 in term 1.
I20260806 16:07:24.780864 23120 leader_election.cc:302] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4ddf2322395047c88574377a755e0cec, e026575bc6a44d48b25303e1b946f322, e2a62dd7d9b940cebe3d52a7e2bc68cf; no voters:
I20260806 16:07:24.781158 23541 raft_consensus.cc:2781] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:07:24.781533 23541 raft_consensus.cc:686] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 1 LEADER]: Becoming Leader. State: Replica: e026575bc6a44d48b25303e1b946f322, State: Running, Role: LEADER
I20260806 16:07:24.781994 23541 consensus_queue.cc:237] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e026575bc6a44d48b25303e1b946f322" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 43665 } } peers { permanent_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46523 } } peers { permanent_uuid: "4ddf2322395047c88574377a755e0cec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 36469 } }
W20260806 16:07:24.782177 23503 tablet.cc:2369] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:07:24.786609 22833 catalog_manager.cc:5495] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 reported cstate change: term changed from 0 to 1, leader changed from <none> to e026575bc6a44d48b25303e1b946f322 (127.8.49.2). New cstate: current_term: 1 leader_uuid: "e026575bc6a44d48b25303e1b946f322" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e026575bc6a44d48b25303e1b946f322" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 43665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ddf2322395047c88574377a755e0cec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 36469 } health_report { overall_health: UNKNOWN } } }
I20260806 16:07:24.804533 23525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.799160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.8.49.2:43665)
0806 16:07:24.799334 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 16:07:24.799338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:07:24.799360 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:07:24.799704 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:07:24.799708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:07:24.799789 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:07:24.799946 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:07:24.800531 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.801127 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.802414 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:07:24.802687 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:07:24.802810 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:07:24.803347 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:07:24.803961 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:07:24.804092 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 16:07:24.804250 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 16:07:24.804326 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 16:07:24.814198 23027 raft_consensus.cc:1261] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec [term 1 FOLLOWER]: Refusing update from remote peer e026575bc6a44d48b25303e1b946f322: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:07:24.814266 23421 raft_consensus.cc:1261] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf [term 1 FOLLOWER]: Refusing update from remote peer e026575bc6a44d48b25303e1b946f322: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:07:24.815454 23541 consensus_queue.cc:1035] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ddf2322395047c88574377a755e0cec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 36469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:07:24.815923 23547 consensus_queue.cc:1035] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a62dd7d9b940cebe3d52a7e2bc68cf" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:07:24.827347 23559 mvcc.cc:205] Tried to move back new op lower bound from 7315588893946183680 to 7315588893830897664. Current Snapshot: MvccSnapshot[applied={T|T < 7315588893946183680}]
I20260806 16:07:24.841583 22832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:07:24.841910 22832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:07:24.845949 22832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 325bd16bafff4e08912a58ac582cd875: Sending DeleteTablet for 3 replicas of tablet 5909c9142d7747df819ab2fd51210b80
I20260806 16:07:24.847311 23394 tablet_service.cc:1514] Processing DeleteTablet for tablet 5909c9142d7747df819ab2fd51210b80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:07:24 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:41838
I20260806 16:07:24.847442 23191 tablet_service.cc:1514] Processing DeleteTablet for tablet 5909c9142d7747df819ab2fd51210b80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:07:24 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:35382
I20260806 16:07:24.847559 23001 tablet_service.cc:1514] Processing DeleteTablet for tablet 5909c9142d7747df819ab2fd51210b80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:07:24 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:52968
I20260806 16:07:24.848058 23569 tablet_replica.cc:330] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: stopping tablet replica
I20260806 16:07:24.848407 23571 tablet_replica.cc:330] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec: stopping tablet replica
I20260806 16:07:24.848589 23569 raft_consensus.cc:2229] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:07:24.848995 23571 raft_consensus.cc:2229] T 5909c9142d7747df819ab2fd51210b80 P 4ddf2322395047c88574377a755e0cec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:07:24.849254 23569 raft_consensus.cc:2258] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:07:24.849753 23570 tablet_replica.cc:330] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: stopping tablet replica
I20260806 16:07:24.849968 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 22921
I20260806 16:07:24.850103 23570 raft_consensus.cc:2229] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:07:24.850471 23570 raft_consensus.cc:2258] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:07:24.851709 23569 ts_tablet_manager.cc:1905] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:07:24.855229 23570 ts_tablet_manager.cc:1905] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:07:24.857328 23569 ts_tablet_manager.cc:1918] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:07:24.857643 23569 log.cc:1196] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/5909c9142d7747df819ab2fd51210b80
I20260806 16:07:24.858136 23569 ts_tablet_manager.cc:1939] T 5909c9142d7747df819ab2fd51210b80 P e026575bc6a44d48b25303e1b946f322: Deleting consensus metadata
I20260806 16:07:24.858769 23570 ts_tablet_manager.cc:1918] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:07:24.858928 23570 log.cc:1196] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/5909c9142d7747df819ab2fd51210b80
I20260806 16:07:24.859318 23570 ts_tablet_manager.cc:1939] T 5909c9142d7747df819ab2fd51210b80 P e2a62dd7d9b940cebe3d52a7e2bc68cf: Deleting consensus metadata
I20260806 16:07:24.860773 22804 catalog_manager.cc:4841] TS e2a62dd7d9b940cebe3d52a7e2bc68cf (127.8.49.3:46523): tablet 5909c9142d7747df819ab2fd51210b80 (table test-table [id=c7a5c628a67f4a0aa154a707d6328611]) successfully deleted
I20260806 16:07:24.863080 22809 catalog_manager.cc:4841] TS e026575bc6a44d48b25303e1b946f322 (127.8.49.2:43665): tablet 5909c9142d7747df819ab2fd51210b80 (table test-table [id=c7a5c628a67f4a0aa154a707d6328611]) successfully deleted
W20260806 16:07:24.871695 22805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:36469 (error 108)
I20260806 16:07:24.871917 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 23096
I20260806 16:07:24.880272 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:07:24.879710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:36469 (local address 127.0.0.1:52978)
0806 16:07:24.879997 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 16:07:24.880137 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:36469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260806 16:07:24.880554 22805 catalog_manager.cc:4584] TS 4ddf2322395047c88574377a755e0cec (127.8.49.1:36469): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5909c9142d7747df819ab2fd51210b80: Network error: Client connection negotiation failed: client connection to 127.8.49.1:36469: connect: Connection refused (error 111)
I20260806 16:07:24.886565 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 23301
I20260806 16:07:24.910756 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 22784
2026-08-06T16:07:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (3250 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:07:25 dist-test-slave-7z32 krb5kdc[23576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:07:25 dist-test-slave-7z32 krb5kdc[23576](info): set up 2 sockets
Aug 06 16:07:25 dist-test-slave-7z32 krb5kdc[23576](info): commencing operation
krb5kdc: starting...
W20260806 16:07:26.343125 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.393s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:07:26 dist-test-slave-7z32 krb5kdc[23576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:07:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:07:26Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.49.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.49.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/rangeradmin_127.8.49.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/rangeradmin_127.8.49.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.49.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.49.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/rangerlookup_127.8.49.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/rangerlookup_127.8.49.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.49.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.49.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/HTTP_127.8.49.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/HTTP_127.8.49.20@KRBTEST.COM.keytab.
I20260806 16:07:26.608990 8388 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrledPc/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/postgres -l logfile start
2026-08-06 16:07:30.460 UTC [23622] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 16:07:30.460 UTC [23622] LOG: listening on IPv4 address "127.8.49.20", port 33491
2026-08-06 16:07:30.465 UTC [23622] LOG: listening on Unix socket "/tmp/.s.PGSQL.33491"
2026-08-06 16:07:30.494 UTC [23626] LOG: database system was shut down at 2026-08-06 16:07:27 UTC
2026-08-06 16:07:30.499 UTC [23622] LOG: database system is ready to accept connections
I20260806 16:07:31.739727 8388 mini_postgres.cc:96] Postgres bound to 33491
2026-08-06 16:07:31.751 UTC [23635] FATAL: database "slave" does not exist
127.8.49.20:33491 - accepting connections
I20260806 16:07:31.753253 8388 mini_ranger.cc:161] Starting Ranger...
I20260806 16:07:31.777001 8388 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 16:07:32.551573 8388 mini_ranger.cc:87] Created ranger Postgres database
I20260806 16:07:32.551699 8388 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-admin
2026-08-06 16:07:32,922 [I] DB FLAVOR :POSTGRES
2026-08-06 16:07:32,923 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:07:32,923 [I] Checking connection..
2026-08-06 16:07:32,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 16:07:33,252 [I] Checking connection passed.
2026-08-06 16:07:33,252 [I] --------- Verifying version history table ---------
2026-08-06 16:07:33,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:07:33,564 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:07:33,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:07:33,892 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:07:33,893 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 16:07:33,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 16:07:34.193 UTC [23733] WARNING: there is no transaction in progress
2026-08-06 16:07:34,204 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 16:07:34,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:07:34,530 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 16:07:34,530 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:07:34,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 16:07:34,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 16:07:35,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7z32.c.gcp-upstream.internal','N') ;"
2026-08-06 16:07:35,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:07:35,887 [I] Table x_portal_user does not exist in database ranger
2026-08-06 16:07:35,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:07:36,239 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 16:07:36,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:07:36,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:07:36,875 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 16:07:36,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 16:07:38.149 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.178 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.210 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.239 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.264 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.293 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.370 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.379 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.398 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.413 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.427 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.445 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.457 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.465 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.474 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:38.483 UTC [23889] WARNING: there is no transaction in progress
2026-08-06 16:07:39,582 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 16:07:39,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 16:07:39,894 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 16:07:39,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:07:40,182 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 16:07:40,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:07:40,510 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 16:07:40,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:07:40,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:07:41,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7z32.c.gcp-upstream.internal';"
2026-08-06 16:07:41,413 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 16:07:41,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 16:07:41,762 [I] DB_PATCHES have already been applied
I20260806 16:07:41.769279 8388 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrledPc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 16:07:41.769506 8388 mini_ranger.cc:193] Using host: 127.8.49.20
I20260806 16:07:41.776522 8388 mini_ranger.cc:227] Ranger admin URL: http://127.8.49.20:39107
Aug 06, 2026 4:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 4:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 4:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 4:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 4:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 16:07:43.221918 8388 mini_ranger.cc:160] Time spent starting Ranger: real 11.469s user 0.000s sys 0.021s
Aug 06, 2026 4:08:10 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 4:08:10 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 4:08:10 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 4:08:10 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 4:08:11 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 16:08:12.236061 8388 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.49.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.49.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/rangerkms_127.8.49.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/rangerkms_127.8.49.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.49.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.49.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.49.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/HTTP_127.8.49.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.49.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/HTTP_127.8.49.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 16:08:12 dist-test-slave-7z32 krb5kdc[23576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032492, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 16:08:12.350989 8388 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 16:08:12.365577 8388 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 16:08:13.511891 8388 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 16:08:13.512040 8388 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-kms
I20260806 16:08:13.512176 8388 mini_ranger_kms.cc:227] Using postgres at 127.8.49.20:33491
2026-08-06 16:08:13,924 [I] DB FLAVOR :POSTGRES
2026-08-06 16:08:13,924 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:08:13,924 [I] Checking connection
2026-08-06 16:08:13,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 16:08:14,251 [I] connection success
2026-08-06 16:08:14,251 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 16:08:14,252 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 16:08:14,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 16:08:14,581 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 16:08:14,581 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:08:14,581 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 16:08:14,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:33491/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 16:08:14,935 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 16:08:15.501523 8388 mini_ranger_kms.cc:310] Created kms service
I20260806 16:08:15.759938 8388 mini_ranger_kms.cc:326] Created kudu user
I20260806 16:08:15.847185 8388 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 16:08:15 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032492, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:08:16.457525 8388 mini_ranger_kms.cc:384] Added ranger policy
I20260806 16:08:16.457705 8388 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 16:08:16.457731 8388 mini_ranger_kms.cc:240] Using host: 127.8.49.20
I20260806 16:08:16.463630 8388 mini_ranger_kms.cc:276] Ranger KMS PID: 24161
I20260806 16:08:16.463804 8388 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.49.20:56089
Aug 06, 2026 4:08:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:08:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:08:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 4:08:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 4:08:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 4:08:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 16:08:19 dist-test-slave-7z32 krb5kdc[23576](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786032499, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.49.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:08:19.465314 8388 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.114s user 0.009s sys 0.382s
I20260806 16:08:19.465497 8388 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 16:08:19.465632 8388 mini_ranger_kms.cc:401] 127.8.49.20:56089/kms/v1/keys
Aug 06 16:08:19 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032499, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.49.20@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
Aug 06 16:08:20 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032499, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.49.20@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
Aug 06 16:08:22 dist-test-slave-7z32 krb5kdc[23576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:08:22.944602 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:39051
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:44821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:39051
--ranger_config_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:56089/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:08:23.103348 24230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:08:23.103840 24230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:08:23.103927 24230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:08:23.108771 24230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:08:23.108939 24230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:08:23.108979 24230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:08:23.109018 24230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:08:23.109053 24230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:08:23.116055 24230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:44821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.49.20:56089/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:39051
--ranger_config_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:39051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:08:23.117772 24230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:08:23.119412 24230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:08:23.132822 24237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:08:23.132908 24235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:08:23.132817 24230 server_base.cc:1034] running on GCE node
W20260806 16:08:23.132982 24239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:08:23.133941 24230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:08:23.135841 24230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:08:23.137319 24230 hybrid_clock.cc:648] HybridClock initialized: now 1786032503137253 us; error 90 us; skew 500 ppm
Aug 06 16:08:23 dist-test-slave-7z32 krb5kdc[23576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:08:23.145318 24230 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:08:23.146708 24230 webserver.cc:460] Webserver started at http://127.8.49.62:41335/ using document root <none> and password file <none>
I20260806 16:08:23.147464 24230 fs_manager.cc:359] Metadata directory not provided
I20260806 16:08:23.147573 24230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:08:23.147846 24230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:08:23 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:08:23.214159 24230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "bd4a749c76fc4cc0ac3b2cb53f7eb7c5"
format_stamp: "Formatted at 2026-08-06 16:08:23 on dist-test-slave-7z32"
server_key: "d1322c13c3a0ff6045b11fdd4b232c51"
server_key_iv: "162b6dd7cb238f772a5b2b5d125913bc"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:23.215116 24230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "bd4a749c76fc4cc0ac3b2cb53f7eb7c5"
format_stamp: "Formatted at 2026-08-06 16:08:23 on dist-test-slave-7z32"
server_key: "d1322c13c3a0ff6045b11fdd4b232c51"
server_key_iv: "162b6dd7cb238f772a5b2b5d125913bc"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:23.222491 24230 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 16:08:23.249013 24248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:08:23.250730 24230 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 16:08:23.250988 24230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "bd4a749c76fc4cc0ac3b2cb53f7eb7c5"
format_stamp: "Formatted at 2026-08-06 16:08:23 on dist-test-slave-7z32"
server_key: "d1322c13c3a0ff6045b11fdd4b232c51"
server_key_iv: "162b6dd7cb238f772a5b2b5d125913bc"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:23.251152 24230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:08:23.270010 24230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:08:23.274065 24230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:08:23.274267 24230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:08:23.284593 24230 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:39051
I20260806 16:08:23.284785 24321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:39051 every 8 connection(s)
I20260806 16:08:23.285982 24230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:08:23.289108 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 24230
I20260806 16:08:23.289292 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:08:23.289700 8388 external_mini_cluster.cc:1428] Setting key fb180639e98ad54a6f9b35f76109067b
I20260806 16:08:23.290968 24323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:08:23.301010 24323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5: Bootstrap starting.
Aug 06 16:08:23 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:08:23.305384 24323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5: Neither blocks nor log segments found. Creating new log.
I20260806 16:08:23.307214 24323 log.cc:824] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5: Log is configured to *not* fsync() on all Append() calls
I20260806 16:08:23.310652 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:23.291753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.8.49.62:39051)
0806 16:08:23.292681 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0806 16:08:23.292705 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:08:23.292765 (+ 60us) server_negotiation.cc:407] Connection header received
0806 16:08:23.293562 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:08:23.293583 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:08:23.294056 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:08:23.294456 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:08:23.295445 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:23.296628 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:23.298203 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:23.298945 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:23.299140 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:23.303091 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:08:23.303100 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:08:23.303123 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:08:23.303174 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:08:23.305661 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:08:23.306315 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:08:23.306322 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:08:23.306329 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:08:23.306482 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:08:23.306796 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:08:23.306799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:08:23.306801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:08:23.307518 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:08:23.307780 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 16:08:23.308677 (+ 897us) server_negotiation.cc:299] Negotiation successful
0806 16:08:23.309273 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":246,"threads_started":1}
I20260806 16:08:23.311556 24323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5: No bootstrap required, opened a new log
I20260806 16:08:23.316428 24323 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd4a749c76fc4cc0ac3b2cb53f7eb7c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39051 } }
I20260806 16:08:23.316804 24323 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:08:23.316864 24323 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4a749c76fc4cc0ac3b2cb53f7eb7c5, State: Initialized, Role: FOLLOWER
I20260806 16:08:23.317497 24323 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd4a749c76fc4cc0ac3b2cb53f7eb7c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39051 } }
I20260806 16:08:23.317669 24323 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:08:23.317724 24323 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:08:23.317824 24323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:08:23.319307 24323 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd4a749c76fc4cc0ac3b2cb53f7eb7c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39051 } }
I20260806 16:08:23.319775 24323 leader_election.cc:302] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd4a749c76fc4cc0ac3b2cb53f7eb7c5; no voters:
I20260806 16:08:23.320268 24323 leader_election.cc:288] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:08:23.320528 24331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:08:23.321055 24331 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [term 1 LEADER]: Becoming Leader. State: Replica: bd4a749c76fc4cc0ac3b2cb53f7eb7c5, State: Running, Role: LEADER
I20260806 16:08:23.321601 24331 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd4a749c76fc4cc0ac3b2cb53f7eb7c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39051 } }
I20260806 16:08:23.322402 24323 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:08:23.323861 24333 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd4a749c76fc4cc0ac3b2cb53f7eb7c5. Latest consensus state: current_term: 1 leader_uuid: "bd4a749c76fc4cc0ac3b2cb53f7eb7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd4a749c76fc4cc0ac3b2cb53f7eb7c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39051 } } }
I20260806 16:08:23.324064 24333 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [sys.catalog]: This master's current role is: LEADER
I20260806 16:08:23.324457 24332 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd4a749c76fc4cc0ac3b2cb53f7eb7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd4a749c76fc4cc0ac3b2cb53f7eb7c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39051 } } }
I20260806 16:08:23.324571 24345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:08:23.324637 24332 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5 [sys.catalog]: This master's current role is: LEADER
I20260806 16:08:23.329216 24345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:08:23.329906 24323 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 16:08:23.345054 24345 catalog_manager.cc:1347] Generated new cluster ID: 2b203a7f25964d56a6f858cd0a0980d2
I20260806 16:08:23.345166 24345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:08:23.359398 24345 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:08:23.360888 24345 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:08:23.367978 24345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5: Generated new TSK 0
I20260806 16:08:23.368992 24345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:08:24.255417 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:56089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39051
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:44821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:08:24.419850 24388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:08:24.420444 24388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:08:24.420629 24388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:08:24.428706 24388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:08:24.428864 24388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:08:24.428970 24388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:08:24.439319 24388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:44821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.49.20:56089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:08:24.441408 24388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:08:24.443331 24388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:08:24.459312 24393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:08:24.459383 24394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:08:24.459754 24396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:08:24.460618 24388 server_base.cc:1034] running on GCE node
I20260806 16:08:24.461313 24388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:08:24.462170 24388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:08:24.463595 24388 hybrid_clock.cc:648] HybridClock initialized: now 1786032504463541 us; error 74 us; skew 500 ppm
Aug 06 16:08:24 dist-test-slave-7z32 krb5kdc[23576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:08:24.468477 24388 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:08:24.470343 24388 webserver.cc:460] Webserver started at http://127.8.49.1:43817/ using document root <none> and password file <none>
I20260806 16:08:24.471433 24388 fs_manager.cc:359] Metadata directory not provided
I20260806 16:08:24.471560 24388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:08:24.471881 24388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:08:24 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:08:24.504287 24388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "fc5f9e718a2644b9b6c275c7bcda20ec"
format_stamp: "Formatted at 2026-08-06 16:08:24 on dist-test-slave-7z32"
server_key: "09ab473132e7fa6255c6bfea76261761"
server_key_iv: "41d3aea940774c747dea7d1d88dcc408"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:24.505581 24388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "fc5f9e718a2644b9b6c275c7bcda20ec"
format_stamp: "Formatted at 2026-08-06 16:08:24 on dist-test-slave-7z32"
server_key: "09ab473132e7fa6255c6bfea76261761"
server_key_iv: "41d3aea940774c747dea7d1d88dcc408"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:24.511857 24388 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 16:08:24.542486 24406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:08:24.545578 24388 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.001s
I20260806 16:08:24.545846 24388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "fc5f9e718a2644b9b6c275c7bcda20ec"
format_stamp: "Formatted at 2026-08-06 16:08:24 on dist-test-slave-7z32"
server_key: "09ab473132e7fa6255c6bfea76261761"
server_key_iv: "41d3aea940774c747dea7d1d88dcc408"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:24.546085 24388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:08:24.573311 24388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:08:24.579481 24388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:08:24.579972 24388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:08:24.581341 24388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:08:24.583519 24388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:08:24.583741 24388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 16:08:24.583914 24388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:08:24.584015 24388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:08:24.599929 24388 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:42707
I20260806 16:08:24.601673 24388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:08:24.603134 24575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:42707 every 8 connection(s)
I20260806 16:08:24.612095 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 24388
I20260806 16:08:24.612287 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:08:24.612679 8388 external_mini_cluster.cc:1428] Setting key 23816d1b18cdd0487fec95c05c0c3d4b
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
Aug 06 16:08:24 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:08:24.646386 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:24.609437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:56477 (local address 127.8.49.62:39051)
0806 16:08:24.609903 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 16:08:24.609911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:08:24.611324 (+ 1413us) server_negotiation.cc:407] Connection header received
0806 16:08:24.624351 (+ 13027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:08:24.624356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:08:24.624473 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:08:24.624609 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:08:24.627558 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:24.629077 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:24.631996 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:24.632459 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:24.632685 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:24.640674 (+ 7989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:08:24.640680 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:08:24.640683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:08:24.640725 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:08:24.643062 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:08:24.644002 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:08:24.644007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:08:24.644010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:08:24.644125 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:08:24.644630 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:08:24.644634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:08:24.644636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:08:24.644941 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:08:24.645050 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 16:08:24.645969 (+ 919us) server_negotiation.cc:299] Negotiation successful
0806 16:08:24.646156 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":171,"threads_started":1}
I20260806 16:08:24.649123 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:24.610049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:39051 (local address 127.8.49.1:56477)
0806 16:08:24.611026 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0806 16:08:24.611130 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0806 16:08:24.613293 (+ 2163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:08:24.625021 (+ 11728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:08:24.625039 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:08:24.625616 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:08:24.627271 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:24.627308 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:24.629582 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:24.629590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:24.631722 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:24.631751 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:24.632892 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:24.632901 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:24.633187 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:24.634450 (+ 1263us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:08:24.634478 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:08:24.640196 (+ 5718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:08:24.643295 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:08:24.643304 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:08:24.643335 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:08:24.643840 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:08:24.644253 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:08:24.644256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:08:24.644259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:08:24.644494 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:08:24.645150 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:08:24.645159 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:08:24.645608 (+ 449us) client_negotiation.cc:769] Sending connection context
0806 16:08:24.646970 (+ 1362us) client_negotiation.cc:240] Negotiation successful
0806 16:08:24.647639 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":242,"threads_started":1}
I20260806 16:08:24.651753 24576 heartbeater.cc:347] Connected to a master server at 127.8.49.62:39051
I20260806 16:08:24.652123 24576 heartbeater.cc:464] Registering TS with master...
I20260806 16:08:24.653592 24576 heartbeater.cc:511] Master 127.8.49.62:39051 requested a full tablet report, sending...
I20260806 16:08:24.656513 24274 ts_manager.cc:194] Registered new tserver with Master: fc5f9e718a2644b9b6c275c7bcda20ec (127.8.49.1:42707)
I20260806 16:08:24.658233 24274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:56477
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:08:24.680027 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:56089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39051
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:44821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:08:24 dist-test-slave-7z32 krb5kdc[23576](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786032504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 16:08:24.888936 24588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:08:24.890059 24588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:08:24.890269 24588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:08:24.895607 24588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:08:24.895753 24588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:08:24.895869 24588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:08:24.902758 24588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:44821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.49.20:56089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:08:24.904670 24588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:08:24.907343 24588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:08:24.924314 24598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:08:24.924280 24595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:08:24.925125 24596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:08:24.925146 24588 server_base.cc:1034] running on GCE node
I20260806 16:08:24.926019 24588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:08:24.927175 24588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:08:24.934732 24588 hybrid_clock.cc:648] HybridClock initialized: now 1786032504932799 us; error 1927 us; skew 500 ppm
Aug 06 16:08:24 dist-test-slave-7z32 krb5kdc[23576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:08:24.940012 24588 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:08:24.941690 24588 webserver.cc:460] Webserver started at http://127.8.49.2:39003/ using document root <none> and password file <none>
I20260806 16:08:24.942806 24588 fs_manager.cc:359] Metadata directory not provided
I20260806 16:08:24.943040 24588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:08:24.943394 24588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:08:24 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:08:24.976224 24588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "372dcffb24f84a5c81055ad40305cd8d"
format_stamp: "Formatted at 2026-08-06 16:08:24 on dist-test-slave-7z32"
server_key: "c7dcd9c209d1ec733b71e04f67828493"
server_key_iv: "09e239a280ec73d9b340652d6190b0e0"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:24.977322 24588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "372dcffb24f84a5c81055ad40305cd8d"
format_stamp: "Formatted at 2026-08-06 16:08:24 on dist-test-slave-7z32"
server_key: "c7dcd9c209d1ec733b71e04f67828493"
server_key_iv: "09e239a280ec73d9b340652d6190b0e0"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:24.985276 24588 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 16:08:25.018317 24605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:08:25.021939 24588 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260806 16:08:25.022215 24588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "372dcffb24f84a5c81055ad40305cd8d"
format_stamp: "Formatted at 2026-08-06 16:08:24 on dist-test-slave-7z32"
server_key: "c7dcd9c209d1ec733b71e04f67828493"
server_key_iv: "09e239a280ec73d9b340652d6190b0e0"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:25.022459 24588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:08:25.050412 24588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:08:25.056208 24588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:08:25.056476 24588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:08:25.057375 24588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:08:25.058815 24588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:08:25.059051 24588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 16:08:25.059175 24588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:08:25.059252 24588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:08:25.073812 24588 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:34987
I20260806 16:08:25.073937 24772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:34987 every 8 connection(s)
I20260806 16:08:25.075165 24588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:08:25.078188 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 24588
I20260806 16:08:25.078411 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:08:25.078768 8388 external_mini_cluster.cc:1428] Setting key edf6f3e823fbc659115bca654da8aeb9
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
Aug 06 16:08:25 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:08:25.099924 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.078775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:45133 (local address 127.8.49.62:39051)
0806 16:08:25.079004 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 16:08:25.079011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:08:25.080212 (+ 1201us) server_negotiation.cc:407] Connection header received
0806 16:08:25.081571 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:08:25.081575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:08:25.081646 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:08:25.081791 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:08:25.084213 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.085144 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.087325 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.087722 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.087818 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.092286 (+ 4468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:08:25.092291 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:08:25.092294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:08:25.092345 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:08:25.095398 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:08:25.096498 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:08:25.096505 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:08:25.096507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:08:25.096678 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:08:25.097586 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:08:25.097590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:08:25.097592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:08:25.097999 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:08:25.098140 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 16:08:25.099404 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0806 16:08:25.099612 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 16:08:25.101929 24776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.079386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:39051 (local address 127.8.49.2:45133)
0806 16:08:25.080016 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0806 16:08:25.080077 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 16:08:25.081280 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:08:25.082309 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:08:25.082326 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:08:25.082931 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:08:25.083918 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.083948 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.085386 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.085390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.087128 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.087144 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.088132 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.088137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.088407 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.089428 (+ 1021us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:08:25.089458 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:08:25.091846 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:08:25.095706 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:08:25.095716 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:08:25.095738 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:08:25.096268 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:08:25.096961 (+ 693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:08:25.096968 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:08:25.096972 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:08:25.097305 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:08:25.098219 (+ 914us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:08:25.098232 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:08:25.098951 (+ 719us) client_negotiation.cc:769] Sending connection context
0806 16:08:25.099301 (+ 350us) client_negotiation.cc:240] Negotiation successful
0806 16:08:25.099876 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":179,"threads_started":1}
I20260806 16:08:25.104506 24773 heartbeater.cc:347] Connected to a master server at 127.8.49.62:39051
I20260806 16:08:25.104969 24773 heartbeater.cc:464] Registering TS with master...
I20260806 16:08:25.105830 24773 heartbeater.cc:511] Master 127.8.49.62:39051 requested a full tablet report, sending...
I20260806 16:08:25.108129 24274 ts_manager.cc:194] Registered new tserver with Master: 372dcffb24f84a5c81055ad40305cd8d (127.8.49.2:34987)
I20260806 16:08:25.109334 24274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:45133
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:08:25.135195 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:56089/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39051
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:44821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:08:25 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
W20260806 16:08:25.369098 24780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:08:25.369861 24780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:08:25.370016 24780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:08:25.377647 24780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:08:25.377909 24780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:08:25.378077 24780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:08:25.388430 24780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:44821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.49.20:56089/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:08:25.391098 24780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:08:25.393296 24780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:08:25.412158 24790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:08:25.413792 24780 server_base.cc:1034] running on GCE node
W20260806 16:08:25.412158 24788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:08:25.414813 24787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:08:25.415623 24780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:08:25.416633 24780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:08:25.418787 24780 hybrid_clock.cc:648] HybridClock initialized: now 1786032505418667 us; error 93 us; skew 500 ppm
Aug 06 16:08:25 dist-test-slave-7z32 krb5kdc[23576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032505, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:08:25.423918 24780 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:08:25.425897 24780 webserver.cc:460] Webserver started at http://127.8.49.3:39381/ using document root <none> and password file <none>
I20260806 16:08:25.426959 24780 fs_manager.cc:359] Metadata directory not provided
I20260806 16:08:25.427138 24780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:08:25.427462 24780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:08:25 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032505, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:08:25.447834 24780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "7b6da8e5b91f4398b17d6117ee5036d6"
format_stamp: "Formatted at 2026-08-06 16:08:25 on dist-test-slave-7z32"
server_key: "39431c37341f15b5baf05841e9187a3c"
server_key_iv: "25102f063b8570daf1948221a33b87be"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:25.448905 24780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "7b6da8e5b91f4398b17d6117ee5036d6"
format_stamp: "Formatted at 2026-08-06 16:08:25 on dist-test-slave-7z32"
server_key: "39431c37341f15b5baf05841e9187a3c"
server_key_iv: "25102f063b8570daf1948221a33b87be"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:25.461637 24780 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.005s sys 0.000s
I20260806 16:08:25.498072 24798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:08:25.501299 24780 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.004s sys 0.000s
I20260806 16:08:25.501562 24780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "7b6da8e5b91f4398b17d6117ee5036d6"
format_stamp: "Formatted at 2026-08-06 16:08:25 on dist-test-slave-7z32"
server_key: "39431c37341f15b5baf05841e9187a3c"
server_key_iv: "25102f063b8570daf1948221a33b87be"
server_key_version: "kuduclusterkey@0"
I20260806 16:08:25.501803 24780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:08:25.552052 24780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:08:25.557644 24780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:08:25.557946 24780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:08:25.559016 24780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:08:25.560531 24780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:08:25.560609 24780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:08:25.560684 24780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:08:25.560719 24780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:08:25.577553 24780 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:41153
I20260806 16:08:25.577646 24962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:41153 every 8 connection(s)
I20260806 16:08:25.579340 24780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:08:25.583446 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 24780
I20260806 16:08:25.583607 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:08:25.583983 8388 external_mini_cluster.cc:1428] Setting key 1369361d1e353f9f90da726bc3325016
Aug 06 16:08:25 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032505, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:08:25.605763 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.582460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:35795 (local address 127.8.49.62:39051)
0806 16:08:25.582767 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 16:08:25.582773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:08:25.584224 (+ 1451us) server_negotiation.cc:407] Connection header received
0806 16:08:25.585942 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:08:25.585948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:08:25.586027 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:08:25.586090 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:08:25.588681 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.589574 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.591892 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.592290 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.592453 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.599110 (+ 6657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:08:25.599116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:08:25.599120 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:08:25.599186 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:08:25.601399 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:08:25.602442 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:08:25.602446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:08:25.602449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:08:25.602578 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:08:25.603159 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:08:25.603163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:08:25.603166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:08:25.603484 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:08:25.603607 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:08:25.605365 (+ 1758us) server_negotiation.cc:299] Negotiation successful
0806 16:08:25.605558 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260806 16:08:25.606207 24966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.583203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:39051 (local address 127.8.49.3:35795)
0806 16:08:25.583964 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 16:08:25.584039 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 16:08:25.585586 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:08:25.586678 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:08:25.586691 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:08:25.587250 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:08:25.588406 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.588435 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.589847 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.589852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.591676 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.591696 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.592568 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.592576 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.592902 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.594010 (+ 1108us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:08:25.594051 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:08:25.596564 (+ 2513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:08:25.601635 (+ 5071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:08:25.601647 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:08:25.601675 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:08:25.602263 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:08:25.602709 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:08:25.602713 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:08:25.602715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:08:25.603014 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:08:25.603637 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:08:25.603646 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:08:25.604130 (+ 484us) client_negotiation.cc:769] Sending connection context
0806 16:08:25.604429 (+ 299us) client_negotiation.cc:240] Negotiation successful
0806 16:08:25.604903 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":238,"threads_started":1}
I20260806 16:08:25.608464 24963 heartbeater.cc:347] Connected to a master server at 127.8.49.62:39051
I20260806 16:08:25.608760 24963 heartbeater.cc:464] Registering TS with master...
I20260806 16:08:25.611285 24963 heartbeater.cc:511] Master 127.8.49.62:39051 requested a full tablet report, sending...
I20260806 16:08:25.613224 24274 ts_manager.cc:194] Registered new tserver with Master: 7b6da8e5b91f4398b17d6117ee5036d6 (127.8.49.3:41153)
I20260806 16:08:25.614148 24274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:35795
I20260806 16:08:25.624138 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:08:25.641237 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.628463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56330 (local address 127.8.49.62:39051)
0806 16:08:25.628613 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 16:08:25.628618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:08:25.628638 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:08:25.628703 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:08:25.628707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:08:25.628772 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:08:25.628911 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:08:25.630058 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.631302 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.632925 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.633706 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.633889 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.636211 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:08:25.636217 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:08:25.636220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:08:25.636268 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:08:25.638458 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:08:25.639348 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:08:25.639353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:08:25.639356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:08:25.639505 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:08:25.639990 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:08:25.639994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:08:25.639996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:08:25.640293 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:08:25.640476 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 16:08:25.640862 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 16:08:25.641036 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:08:25.647876 24274 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:08:25.653877 24274 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:08:25.663379 24576 heartbeater.cc:503] Master 127.8.49.62:39051 was elected leader, sending a full tablet report...
I20260806 16:08:25.688796 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.673619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:42707 (local address 127.0.0.1:34774)
0806 16:08:25.675069 (+ 1450us) negotiation.cc:107] Waiting for socket to connect
0806 16:08:25.675085 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:08:25.675220 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:08:25.676410 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:08:25.676419 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:08:25.676453 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:08:25.677305 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.677316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.680680 (+ 3364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.680686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.682960 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.682994 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.685930 (+ 2936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.685934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.686280 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.686363 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 16:08:25.686541 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 16:08:25.686991 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1349,"thread_start_us":188,"threads_started":1}
I20260806 16:08:25.690256 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.673771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.8.49.1:42707)
0806 16:08:25.675459 (+ 1688us) server_negotiation.cc:206] Beginning negotiation
0806 16:08:25.675471 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:08:25.675502 (+ 31us) server_negotiation.cc:407] Connection header received
0806 16:08:25.675614 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:08:25.675622 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:08:25.675957 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:08:25.676150 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:08:25.677540 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.680307 (+ 2767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.683277 (+ 2970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.685449 (+ 2172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.685609 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.685836 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 16:08:25.689822 (+ 3986us) server_negotiation.cc:299] Negotiation successful
0806 16:08:25.690031 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1550,"thread_start_us":225,"threads_started":1}
I20260806 16:08:25.690678 24984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.670564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:34987 (local address 127.0.0.1:58984)
0806 16:08:25.674695 (+ 4131us) negotiation.cc:107] Waiting for socket to connect
0806 16:08:25.674797 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0806 16:08:25.675087 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:08:25.684448 (+ 9361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:08:25.684452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:08:25.684484 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:08:25.684624 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.684708 (+ 84us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.686620 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.686623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.688418 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.688435 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.690142 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.690146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.690269 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.690305 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 16:08:25.690362 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 16:08:25.690484 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3792,"thread_start_us":154,"threads_started":1}
I20260806 16:08:25.693532 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.670848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58984 (local address 127.8.49.2:34987)
0806 16:08:25.683240 (+ 12392us) server_negotiation.cc:206] Beginning negotiation
0806 16:08:25.683251 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:08:25.683302 (+ 51us) server_negotiation.cc:407] Connection header received
0806 16:08:25.683437 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:08:25.683445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:08:25.684024 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:08:25.684498 (+ 474us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:08:25.684829 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.686449 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.688675 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.689958 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.692892 (+ 2934us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.693126 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 16:08:25.693204 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 16:08:25.693334 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12159,"thread_start_us":159,"threads_started":1}
I20260806 16:08:25.695374 24462 tablet_service.cc:1467] Processing CreateTablet for tablet a004980d0e7b463da4482bb927c2687b (DEFAULT_TABLE table=test-table [id=1aa40609ffe04423a21da05359ab0e53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:08:25.698786 24462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a004980d0e7b463da4482bb927c2687b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:08:25.698887 24680 tablet_service.cc:1467] Processing CreateTablet for tablet a004980d0e7b463da4482bb927c2687b (DEFAULT_TABLE table=test-table [id=1aa40609ffe04423a21da05359ab0e53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:08:25.701308 24680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a004980d0e7b463da4482bb927c2687b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:08:25.713604 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.676032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:41153 (local address 127.0.0.1:58776)
0806 16:08:25.683519 (+ 7487us) negotiation.cc:107] Waiting for socket to connect
0806 16:08:25.683543 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 16:08:25.684148 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:08:25.689315 (+ 5167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:08:25.689320 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:08:25.689348 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:08:25.689511 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.689519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.691255 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.691261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.703176 (+ 11915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.703203 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.713012 (+ 9809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.713019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.713165 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.713211 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 16:08:25.713291 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 16:08:25.713399 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7306,"thread_start_us":154,"threads_started":1}
I20260806 16:08:25.714985 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.676636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58776 (local address 127.8.49.3:41153)
0806 16:08:25.677185 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 16:08:25.677200 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:08:25.683996 (+ 6796us) server_negotiation.cc:407] Connection header received
0806 16:08:25.684460 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:08:25.684470 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:08:25.688719 (+ 4249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:08:25.688975 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:08:25.689690 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.690811 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.710980 (+ 20169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.712690 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.714273 (+ 1583us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.714529 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 16:08:25.714636 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 16:08:25.714770 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":206,"threads_started":1}
I20260806 16:08:25.723057 24988 tablet_bootstrap.cc:492] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec: Bootstrap starting.
I20260806 16:08:25.726553 24861 tablet_service.cc:1467] Processing CreateTablet for tablet a004980d0e7b463da4482bb927c2687b (DEFAULT_TABLE table=test-table [id=1aa40609ffe04423a21da05359ab0e53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:08:25.728278 24989 tablet_bootstrap.cc:492] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: Bootstrap starting.
I20260806 16:08:25.729583 24861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a004980d0e7b463da4482bb927c2687b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:08:25.733035 24989 tablet_bootstrap.cc:654] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: Neither blocks nor log segments found. Creating new log.
I20260806 16:08:25.734107 24988 tablet_bootstrap.cc:654] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec: Neither blocks nor log segments found. Creating new log.
I20260806 16:08:25.735738 24988 log.cc:824] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec: Log is configured to *not* fsync() on all Append() calls
I20260806 16:08:25.745513 24991 tablet_bootstrap.cc:492] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: Bootstrap starting.
I20260806 16:08:25.745558 24989 log.cc:824] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:08:25.751782 24991 tablet_bootstrap.cc:654] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: Neither blocks nor log segments found. Creating new log.
I20260806 16:08:25.753516 24991 log.cc:824] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: Log is configured to *not* fsync() on all Append() calls
I20260806 16:08:25.754550 24989 tablet_bootstrap.cc:492] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: No bootstrap required, opened a new log
I20260806 16:08:25.755127 24989 ts_tablet_manager.cc:1397] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: Time spent bootstrapping tablet: real 0.028s user 0.011s sys 0.000s
I20260806 16:08:25.757191 24991 tablet_bootstrap.cc:492] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: No bootstrap required, opened a new log
I20260806 16:08:25.757513 24991 ts_tablet_manager.cc:1397] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260806 16:08:25.763810 24991 raft_consensus.cc:348] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372dcffb24f84a5c81055ad40305cd8d" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34987 } } peers { permanent_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 42707 } } peers { permanent_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 41153 } }
I20260806 16:08:25.764430 24991 raft_consensus.cc:374] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:08:25.764545 24991 raft_consensus.cc:729] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b6da8e5b91f4398b17d6117ee5036d6, State: Initialized, Role: FOLLOWER
I20260806 16:08:25.765661 24991 consensus_queue.cc:260] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372dcffb24f84a5c81055ad40305cd8d" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34987 } } peers { permanent_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 42707 } } peers { permanent_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 41153 } }
I20260806 16:08:25.766573 24988 tablet_bootstrap.cc:492] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec: No bootstrap required, opened a new log
I20260806 16:08:25.767019 24988 ts_tablet_manager.cc:1397] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec: Time spent bootstrapping tablet: real 0.048s user 0.009s sys 0.007s
I20260806 16:08:25.767086 24989 raft_consensus.cc:348] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372dcffb24f84a5c81055ad40305cd8d" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34987 } } peers { permanent_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 42707 } } peers { permanent_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 41153 } }
I20260806 16:08:25.767534 24989 raft_consensus.cc:374] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:08:25.767639 24989 raft_consensus.cc:729] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 372dcffb24f84a5c81055ad40305cd8d, State: Initialized, Role: FOLLOWER
I20260806 16:08:25.768352 24991 ts_tablet_manager.cc:1428] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260806 16:08:25.768388 24989 consensus_queue.cc:260] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372dcffb24f84a5c81055ad40305cd8d" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34987 } } peers { permanent_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 42707 } } peers { permanent_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 41153 } }
I20260806 16:08:25.769053 24963 heartbeater.cc:503] Master 127.8.49.62:39051 was elected leader, sending a full tablet report...
I20260806 16:08:25.770223 24989 ts_tablet_manager.cc:1428] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20260806 16:08:25.770812 24773 heartbeater.cc:503] Master 127.8.49.62:39051 was elected leader, sending a full tablet report...
I20260806 16:08:25.772393 24988 raft_consensus.cc:348] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372dcffb24f84a5c81055ad40305cd8d" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34987 } } peers { permanent_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 42707 } } peers { permanent_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 41153 } }
I20260806 16:08:25.772850 24988 raft_consensus.cc:374] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:08:25.772921 24988 raft_consensus.cc:729] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc5f9e718a2644b9b6c275c7bcda20ec, State: Initialized, Role: FOLLOWER
I20260806 16:08:25.773870 24988 consensus_queue.cc:260] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372dcffb24f84a5c81055ad40305cd8d" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34987 } } peers { permanent_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 42707 } } peers { permanent_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 41153 } }
I20260806 16:08:25.776270 24988 ts_tablet_manager.cc:1428] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec: Time spent starting tablet: real 0.009s user 0.006s sys 0.002s
I20260806 16:08:25.819461 24996 raft_consensus.cc:482] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:08:25.819852 24996 raft_consensus.cc:504] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372dcffb24f84a5c81055ad40305cd8d" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34987 } } peers { permanent_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 42707 } } peers { permanent_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 41153 } }
I20260806 16:08:25.823014 24996 leader_election.cc:288] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc5f9e718a2644b9b6c275c7bcda20ec (127.8.49.1:42707), 7b6da8e5b91f4398b17d6117ee5036d6 (127.8.49.3:41153)
I20260806 16:08:25.831020 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.824032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:49085 (local address 127.8.49.1:42707)
0806 16:08:25.824364 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0806 16:08:25.824372 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:08:25.824816 (+ 444us) server_negotiation.cc:407] Connection header received
0806 16:08:25.825624 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:08:25.825630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:08:25.825696 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:08:25.825748 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:08:25.826376 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.827356 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.829329 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.830014 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.830103 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.830192 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:08:25.830657 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 16:08:25.830755 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 16:08:25.831307 24964 tablet.cc:2369] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:08:25.832247 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.824331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:46181 (local address 127.8.49.3:41153)
0806 16:08:25.824564 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 16:08:25.824570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:08:25.824782 (+ 212us) server_negotiation.cc:407] Connection header received
0806 16:08:25.825129 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:08:25.825136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:08:25.825216 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:08:25.825281 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:08:25.826123 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.827133 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.829987 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.830711 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.830856 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.830960 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 16:08:25.831942 (+ 982us) server_negotiation.cc:299] Negotiation successful
0806 16:08:25.832070 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 16:08:25.833091 24775 tablet.cc:2369] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:08:25.833069 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.823571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:41153 (local address 127.8.49.2:46181)
0806 16:08:25.824642 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0806 16:08:25.824667 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 16:08:25.824934 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:08:25.825511 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:08:25.825516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:08:25.825542 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:08:25.825738 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.825746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.827399 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.827403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.829735 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.829755 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.831624 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.831628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.831756 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.831789 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 16:08:25.832717 (+ 928us) client_negotiation.cc:240] Negotiation successful
0806 16:08:25.832850 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"thread_start_us":239,"threads_started":1}
I20260806 16:08:25.834527 24999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.823922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:42707 (local address 127.8.49.2:49085)
0806 16:08:25.824670 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0806 16:08:25.824688 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:08:25.825000 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:08:25.825913 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:08:25.825915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:08:25.825991 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:08:25.826236 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.826246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.827518 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.827521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.829135 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:08:25.829151 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.830346 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.830350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:08:25.830506 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.830531 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 16:08:25.834155 (+ 3624us) client_negotiation.cc:240] Negotiation successful
0806 16:08:25.834306 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":184,"threads_started":1}
I20260806 16:08:25.834681 24898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a004980d0e7b463da4482bb927c2687b" candidate_uuid: "372dcffb24f84a5c81055ad40305cd8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" is_pre_election: true
I20260806 16:08:25.835214 24898 raft_consensus.cc:2454] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 372dcffb24f84a5c81055ad40305cd8d in term 0.
I20260806 16:08:25.835973 24607 leader_election.cc:302] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 372dcffb24f84a5c81055ad40305cd8d, 7b6da8e5b91f4398b17d6117ee5036d6; no voters:
I20260806 16:08:25.835995 24478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a004980d0e7b463da4482bb927c2687b" candidate_uuid: "372dcffb24f84a5c81055ad40305cd8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" is_pre_election: true
Aug 06 16:08:25 dist-test-slave-7z32 krb5kdc[23576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032504, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:08:25.836444 24478 raft_consensus.cc:2454] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 372dcffb24f84a5c81055ad40305cd8d in term 0.
I20260806 16:08:25.836457 24996 raft_consensus.cc:2781] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:08:25.836568 24996 raft_consensus.cc:482] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:08:25.836644 24996 raft_consensus.cc:3037] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:08:25.838626 24996 raft_consensus.cc:504] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372dcffb24f84a5c81055ad40305cd8d" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34987 } } peers { permanent_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 42707 } } peers { permanent_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 41153 } }
I20260806 16:08:25.839324 24996 leader_election.cc:288] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [CANDIDATE]: Term 1 election: Requested vote from peers fc5f9e718a2644b9b6c275c7bcda20ec (127.8.49.1:42707), 7b6da8e5b91f4398b17d6117ee5036d6 (127.8.49.3:41153)
I20260806 16:08:25.840138 24478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a004980d0e7b463da4482bb927c2687b" candidate_uuid: "372dcffb24f84a5c81055ad40305cd8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec"
I20260806 16:08:25.840360 24478 raft_consensus.cc:3037] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:08:25.842450 24478 raft_consensus.cc:2454] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 372dcffb24f84a5c81055ad40305cd8d in term 1.
I20260806 16:08:25.843487 24609 leader_election.cc:302] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 372dcffb24f84a5c81055ad40305cd8d, fc5f9e718a2644b9b6c275c7bcda20ec; no voters:
I20260806 16:08:25.843596 24898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a004980d0e7b463da4482bb927c2687b" candidate_uuid: "372dcffb24f84a5c81055ad40305cd8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b6da8e5b91f4398b17d6117ee5036d6"
I20260806 16:08:25.843811 24898 raft_consensus.cc:3037] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:08:25.846230 24898 raft_consensus.cc:2454] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 372dcffb24f84a5c81055ad40305cd8d in term 1.
I20260806 16:08:25.848474 24996 raft_consensus.cc:2781] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:08:25.848881 24996 raft_consensus.cc:686] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 1 LEADER]: Becoming Leader. State: Replica: 372dcffb24f84a5c81055ad40305cd8d, State: Running, Role: LEADER
I20260806 16:08:25.849448 24996 consensus_queue.cc:237] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372dcffb24f84a5c81055ad40305cd8d" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34987 } } peers { permanent_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 42707 } } peers { permanent_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 41153 } }
I20260806 16:08:25.854997 24274 catalog_manager.cc:5495] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d reported cstate change: term changed from 0 to 1, leader changed from <none> to 372dcffb24f84a5c81055ad40305cd8d (127.8.49.2). New cstate: current_term: 1 leader_uuid: "372dcffb24f84a5c81055ad40305cd8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372dcffb24f84a5c81055ad40305cd8d" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 42707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 41153 } health_report { overall_health: UNKNOWN } } }
W20260806 16:08:25.857494 24577 tablet.cc:2369] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:08:25.889355 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.880139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58992 (local address 127.8.49.2:34987)
0806 16:08:25.880342 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 16:08:25.880347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:08:25.880366 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:08:25.880503 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:08:25.880507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:08:25.880603 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:08:25.880767 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:08:25.881833 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.882886 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.885084 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:08:25.885603 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:08:25.885790 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:08:25.887105 (+ 1315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:08:25.887947 (+ 842us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:08:25.888104 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 16:08:25.888836 (+ 732us) server_negotiation.cc:299] Negotiation successful
0806 16:08:25.888979 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 16:08:25.905790 24478 raft_consensus.cc:1261] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec [term 1 FOLLOWER]: Refusing update from remote peer 372dcffb24f84a5c81055ad40305cd8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:08:25.905944 24898 raft_consensus.cc:1261] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6 [term 1 FOLLOWER]: Refusing update from remote peer 372dcffb24f84a5c81055ad40305cd8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:08:25.906956 24996 consensus_queue.cc:1035] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc5f9e718a2644b9b6c275c7bcda20ec" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 42707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:08:25.907402 25004 consensus_queue.cc:1035] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b6da8e5b91f4398b17d6117ee5036d6" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 41153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:08:25.925637 25008 mvcc.cc:205] Tried to move back new op lower bound from 7315589144175734784 to 7315589143964250112. Current Snapshot: MvccSnapshot[applied={T|T < 7315589144175734784}]
I20260806 16:08:25.928088 25009 mvcc.cc:205] Tried to move back new op lower bound from 7315589144175734784 to 7315589143964250112. Current Snapshot: MvccSnapshot[applied={T|T < 7315589144175734784}]
I20260806 16:08:25.932094 25010 mvcc.cc:205] Tried to move back new op lower bound from 7315589144175734784 to 7315589143964250112. Current Snapshot: MvccSnapshot[applied={T|T < 7315589144175734784}]
I20260806 16:08:25.954509 24261 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:08:25.954751 24261 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:08:25.958614 24261 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd4a749c76fc4cc0ac3b2cb53f7eb7c5: Sending DeleteTablet for 3 replicas of tablet a004980d0e7b463da4482bb927c2687b
I20260806 16:08:25.960001 24680 tablet_service.cc:1514] Processing DeleteTablet for tablet a004980d0e7b463da4482bb927c2687b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:08:25 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:58984
I20260806 16:08:25.960047 24861 tablet_service.cc:1514] Processing DeleteTablet for tablet a004980d0e7b463da4482bb927c2687b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:08:25 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:58776
I20260806 16:08:25.961061 24462 tablet_service.cc:1514] Processing DeleteTablet for tablet a004980d0e7b463da4482bb927c2687b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:08:25 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:34774
I20260806 16:08:25.962299 25020 tablet_replica.cc:330] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec: stopping tablet replica
I20260806 16:08:25.962415 25019 tablet_replica.cc:330] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: stopping tablet replica
I20260806 16:08:25.962816 25020 raft_consensus.cc:2229] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:08:25.962985 25019 raft_consensus.cc:2229] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:08:25.963245 25018 tablet_replica.cc:330] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: stopping tablet replica
I20260806 16:08:25.963735 25020 raft_consensus.cc:2258] T a004980d0e7b463da4482bb927c2687b P fc5f9e718a2644b9b6c275c7bcda20ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:08:25.963850 25019 raft_consensus.cc:2258] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:08:25.964228 25018 raft_consensus.cc:2229] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:08:25.964841 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 24388
I20260806 16:08:25.965240 25018 raft_consensus.cc:2258] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:08:25.972790 25018 ts_tablet_manager.cc:1905] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:08:25.976483 25018 ts_tablet_manager.cc:1918] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:08:25.976729 25018 log.cc:1196] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/a004980d0e7b463da4482bb927c2687b
I20260806 16:08:25.977303 25018 ts_tablet_manager.cc:1939] T a004980d0e7b463da4482bb927c2687b P 372dcffb24f84a5c81055ad40305cd8d: Deleting consensus metadata
I20260806 16:08:25.979112 24249 catalog_manager.cc:4841] TS 372dcffb24f84a5c81055ad40305cd8d (127.8.49.2:34987): tablet a004980d0e7b463da4482bb927c2687b (table test-table [id=1aa40609ffe04423a21da05359ab0e53]) successfully deleted
I20260806 16:08:25.980244 25019 ts_tablet_manager.cc:1905] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 16:08:25.981500 24251 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:42707 (error 108)
I20260806 16:08:25.983841 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:08:25.983363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:42707 (local address 127.0.0.1:34782)
0806 16:08:25.983632 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 16:08:25.983721 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260806 16:08:25.984072 24251 catalog_manager.cc:4584] TS fc5f9e718a2644b9b6c275c7bcda20ec (127.8.49.1:42707): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a004980d0e7b463da4482bb927c2687b: Network error: Client connection negotiation failed: client connection to 127.8.49.1:42707: connect: Connection refused (error 111)
I20260806 16:08:25.985905 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 24588
I20260806 16:08:25.993153 25019 ts_tablet_manager.cc:1918] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:08:25.993374 25019 log.cc:1196] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/a004980d0e7b463da4482bb927c2687b
I20260806 16:08:25.993772 25019 ts_tablet_manager.cc:1939] T a004980d0e7b463da4482bb927c2687b P 7b6da8e5b91f4398b17d6117ee5036d6: Deleting consensus metadata
I20260806 16:08:25.998399 24250 catalog_manager.cc:4841] TS 7b6da8e5b91f4398b17d6117ee5036d6 (127.8.49.3:41153): tablet a004980d0e7b463da4482bb927c2687b (table test-table [id=1aa40609ffe04423a21da05359ab0e53]) successfully deleted
I20260806 16:08:26.008731 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 24780
I20260806 16:08:26.036737 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 24230
I20260806 16:08:26.059152 8388 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 16:08:26.407836 8388 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 16:08:26.408092 8388 mini_ranger.cc:66] Stopping Ranger...
I20260806 16:08:26.776712 8388 mini_ranger.cc:68] Stopped Ranger
2026-08-06 16:08:26.777 UTC [23622] LOG: received smart shutdown request
2026-08-06 16:08:26.782 UTC [23622] LOG: background worker "logical replication launcher" (PID 23633) exited with exit code 1
2026-08-06 16:08:26.783 UTC [23627] LOG: shutting down
2026-08-06 16:08:27.057 UTC [23622] LOG: database system is shut down
2026-08-06T16:08:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (62265 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:08:27 dist-test-slave-7z32 krb5kdc[25034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:08:27 dist-test-slave-7z32 krb5kdc[25034](info): set up 2 sockets
Aug 06 16:08:27 dist-test-slave-7z32 krb5kdc[25034](info): commencing operation
krb5kdc: starting...
W20260806 16:08:28.567901 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.352s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:08:28 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:08:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:08:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.49.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.49.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/rangeradmin_127.8.49.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/rangeradmin_127.8.49.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.49.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.49.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/rangerlookup_127.8.49.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/rangerlookup_127.8.49.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.49.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.49.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/HTTP_127.8.49.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/HTTP_127.8.49.20@KRBTEST.COM.keytab.
I20260806 16:08:28.856143 8388 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrledPc/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/postgres -l logfile start
2026-08-06 16:08:32.606 UTC [25074] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 16:08:32.606 UTC [25074] LOG: listening on IPv4 address "127.8.49.20", port 55115
2026-08-06 16:08:32.611 UTC [25074] LOG: listening on Unix socket "/tmp/.s.PGSQL.55115"
2026-08-06 16:08:32.658 UTC [25078] LOG: database system was shut down at 2026-08-06 16:08:30 UTC
2026-08-06 16:08:32.663 UTC [25074] LOG: database system is ready to accept connections
I20260806 16:08:33.855878 8388 mini_postgres.cc:96] Postgres bound to 55115
2026-08-06 16:08:33.870 UTC [25090] FATAL: database "slave" does not exist
127.8.49.20:55115 - accepting connections
I20260806 16:08:33.871309 8388 mini_ranger.cc:161] Starting Ranger...
I20260806 16:08:33.902961 8388 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 16:08:34.633827 8388 mini_ranger.cc:87] Created ranger Postgres database
I20260806 16:08:34.634060 8388 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-admin
2026-08-06 16:08:34,988 [I] DB FLAVOR :POSTGRES
2026-08-06 16:08:34,989 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:08:34,989 [I] Checking connection..
2026-08-06 16:08:34,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 16:08:35,294 [I] Checking connection passed.
2026-08-06 16:08:35,294 [I] --------- Verifying version history table ---------
2026-08-06 16:08:35,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:08:35,609 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:08:35,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:08:35,921 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:08:35,921 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 16:08:35,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 16:08:36.234 UTC [25172] WARNING: there is no transaction in progress
2026-08-06 16:08:36,245 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 16:08:36,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:08:36,570 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 16:08:36,571 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:08:36,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 16:08:36,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 16:08:37,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7z32.c.gcp-upstream.internal','N') ;"
2026-08-06 16:08:37,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:08:37,797 [I] Table x_portal_user does not exist in database ranger
2026-08-06 16:08:37,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:08:38,117 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 16:08:38,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:08:38,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:08:38,779 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 16:08:38,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 16:08:40.249 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.276 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.304 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.325 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.349 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.373 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.462 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.471 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.493 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.508 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.522 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.540 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.554 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.562 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.573 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:40.583 UTC [25324] WARNING: there is no transaction in progress
2026-08-06 16:08:41,666 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 16:08:41,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 16:08:41,974 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 16:08:41,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:08:42,295 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 16:08:42,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:08:42,591 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 16:08:42,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:08:42,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:08:43,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7z32.c.gcp-upstream.internal';"
2026-08-06 16:08:43,527 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 16:08:43,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 16:08:43,823 [I] DB_PATCHES have already been applied
I20260806 16:08:43.830768 8388 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrledPc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 16:08:43.830955 8388 mini_ranger.cc:193] Using host: 127.8.49.20
I20260806 16:08:43.837638 8388 mini_ranger.cc:227] Ranger admin URL: http://127.8.49.20:54345
Aug 06, 2026 4:08:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:08:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:08:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 4:08:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 4:08:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 4:08:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 4:08:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 16:08:45.293970 8388 mini_ranger.cc:160] Time spent starting Ranger: real 11.423s user 0.003s sys 0.016s
Aug 06, 2026 4:09:11 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 4:09:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 4:09:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 4:09:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 4:09:12 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 16:09:13.466059 8388 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.49.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.49.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/rangerkms_127.8.49.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.49.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/rangerkms_127.8.49.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.49.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.49.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.49.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/HTTP_127.8.49.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.49.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/HTTP_127.8.49.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 16:09:13 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032553, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 16:09:13.576524 8388 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 16:09:13.591979 8388 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 16:09:14.638046 8388 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 16:09:14.638201 8388 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-kms
I20260806 16:09:14.638227 8388 mini_ranger_kms.cc:227] Using postgres at 127.8.49.20:55115
2026-08-06 16:09:14,904 [I] DB FLAVOR :POSTGRES
2026-08-06 16:09:14,904 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:09:14,904 [I] Checking connection
2026-08-06 16:09:14,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 16:09:15,235 [I] connection success
2026-08-06 16:09:15,235 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 16:09:15,235 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 16:09:15,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 16:09:15,604 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 16:09:15,604 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:09:15,604 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 16:09:15,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.49.20:55115/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 16:09:16,016 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 16:09:16.669667 8388 mini_ranger_kms.cc:310] Created kms service
I20260806 16:09:16.889112 8388 mini_ranger_kms.cc:326] Created kudu user
I20260806 16:09:16.946615 8388 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 16:09:16 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032553, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:09:17.449100 8388 mini_ranger_kms.cc:384] Added ranger policy
I20260806 16:09:17.449446 8388 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrledPc/build/debug/bin/postgresql.jar:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrledPc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 16:09:17.449468 8388 mini_ranger_kms.cc:240] Using host: 127.8.49.20
I20260806 16:09:17.455967 8388 mini_ranger_kms.cc:276] Ranger KMS PID: 25587
I20260806 16:09:17.456115 8388 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.49.20:33883
Aug 06, 2026 4:09:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:09:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:09:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 4:09:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 4:09:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 4:09:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 16:09:19 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786032559, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.49.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:20.441258 8388 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.865s user 0.007s sys 0.242s
I20260806 16:09:20.441395 8388 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 16:09:20.441488 8388 mini_ranger_kms.cc:401] 127.8.49.20:33883/kms/v1/keys
Aug 06 16:09:20 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032559, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.49.20@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
Aug 06 16:09:21 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032559, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.49.20@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
Aug 06 16:09:24 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:09:24.735193 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42957
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:42957
--ranger_config_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:33883/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:24.887775 25657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:24.888136 25657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:24.888180 25657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:24.892704 25657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:09:24.892804 25657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:24.892824 25657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:24.892843 25657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:09:24.892858 25657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:09:24.900485 25657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.49.20:33883/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:42957
--ranger_config_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:24.902313 25657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:24.903872 25657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:09:24.917337 25657 server_base.cc:1034] running on GCE node
W20260806 16:09:24.917326 25666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:24.917363 25668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:24.917308 25665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:24.919431 25657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:24.921255 25657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:24.922605 25657 hybrid_clock.cc:648] HybridClock initialized: now 1786032564922564 us; error 69 us; skew 500 ppm
Aug 06 16:09:24 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032564, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:24.927935 25657 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:09:24.929474 25657 webserver.cc:460] Webserver started at http://127.8.49.62:34103/ using document root <none> and password file <none>
I20260806 16:09:24.930375 25657 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:24.930464 25657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:24.930802 25657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:09:24 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032564, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:09:24.993713 25657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "00011490fd234c21a11fa1a23ad72c63"
format_stamp: "Formatted at 2026-08-06 16:09:24 on dist-test-slave-7z32"
server_key: "3352a15c06235dca4f9e050b5c65b917"
server_key_iv: "6a09a6072211522071ac0e144cd4e1d5"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:24.995198 25657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "00011490fd234c21a11fa1a23ad72c63"
format_stamp: "Formatted at 2026-08-06 16:09:24 on dist-test-slave-7z32"
server_key: "3352a15c06235dca4f9e050b5c65b917"
server_key_iv: "6a09a6072211522071ac0e144cd4e1d5"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:25.005880 25657 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.009s sys 0.000s
I20260806 16:09:25.029281 25677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:25.031790 25657 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:09:25.032042 25657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "00011490fd234c21a11fa1a23ad72c63"
format_stamp: "Formatted at 2026-08-06 16:09:24 on dist-test-slave-7z32"
server_key: "3352a15c06235dca4f9e050b5c65b917"
server_key_iv: "6a09a6072211522071ac0e144cd4e1d5"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:25.032261 25657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:25.057225 25657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:25.062955 25657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:25.063231 25657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:25.073225 25657 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:42957
I20260806 16:09:25.073330 25750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:42957 every 8 connection(s)
I20260806 16:09:25.074546 25657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:09:25.079252 25751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:25.080518 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 25657
I20260806 16:09:25.080693 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:09:25.081125 8388 external_mini_cluster.cc:1428] Setting key 19788b762c0977e065b42f21764f933d
I20260806 16:09:25.088531 25751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63: Bootstrap starting.
I20260806 16:09:25.092042 25751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63: Neither blocks nor log segments found. Creating new log.
Aug 06 16:09:25 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:25.093400 25751 log.cc:824] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:25.096840 25751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63: No bootstrap required, opened a new log
I20260806 16:09:25.102020 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:25.083514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52744 (local address 127.8.49.62:42957)
0806 16:09:25.084256 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:25.084279 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:25.084330 (+ 51us) server_negotiation.cc:407] Connection header received
0806 16:09:25.085273 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:25.085296 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:25.085750 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:25.086184 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:25.087210 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:25.088318 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:25.089335 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:25.089881 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:25.090082 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:25.093770 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:25.093778 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:25.093799 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:25.093840 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:25.097420 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:25.098482 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:25.098490 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:25.098497 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:25.098702 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:25.099217 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:25.099221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:25.099222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:25.099805 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:25.100002 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:25.100571 (+ 569us) server_negotiation.cc:299] Negotiation successful
0806 16:09:25.101030 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":226,"threads_started":1}
I20260806 16:09:25.101820 25751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } }
I20260806 16:09:25.102253 25751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:25.102331 25751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00011490fd234c21a11fa1a23ad72c63, State: Initialized, Role: FOLLOWER
I20260806 16:09:25.103104 25751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } }
I20260806 16:09:25.103273 25751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:09:25.103374 25751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:09:25.103498 25751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:25.105240 25751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } }
I20260806 16:09:25.105764 25751 leader_election.cc:302] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00011490fd234c21a11fa1a23ad72c63; no voters:
I20260806 16:09:25.106284 25751 leader_election.cc:288] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:09:25.106552 25757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:09:25.107031 25757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 1 LEADER]: Becoming Leader. State: Replica: 00011490fd234c21a11fa1a23ad72c63, State: Running, Role: LEADER
I20260806 16:09:25.107600 25757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } }
I20260806 16:09:25.108196 25751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:09:25.109927 25760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00011490fd234c21a11fa1a23ad72c63. Latest consensus state: current_term: 1 leader_uuid: "00011490fd234c21a11fa1a23ad72c63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } } }
I20260806 16:09:25.110112 25760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:25.110455 25758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00011490fd234c21a11fa1a23ad72c63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } } }
I20260806 16:09:25.110546 25758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:25.111524 25751 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 16:09:25.111598 25770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:09:25.119408 25770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:09:25.129148 25770 catalog_manager.cc:1347] Generated new cluster ID: ff53f5c3804542f49e9240c83406dfd2
I20260806 16:09:25.129382 25770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:09:25.162815 25770 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:09:25.164388 25770 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:09:25.177232 25770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63: Generated new TSK 0
I20260806 16:09:25.178320 25770 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:09:26.136286 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:33883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:42957
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:26.321699 25813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:26.322216 25813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:26.322291 25813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:26.329350 25813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:26.329479 25813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:26.329579 25813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:09:26.341555 25813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.49.20:33883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:42957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:26.343456 25813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:26.345129 25813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:26.362489 25825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:26.362522 25829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:26.363287 25826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:26.364051 25813 server_base.cc:1034] running on GCE node
I20260806 16:09:26.364810 25813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:26.367018 25813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:26.374972 25813 hybrid_clock.cc:648] HybridClock initialized: now 1786032566374765 us; error 99 us; skew 500 ppm
Aug 06 16:09:26 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032566, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:26.379753 25813 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:09:26.381342 25813 webserver.cc:460] Webserver started at http://127.8.49.1:34879/ using document root <none> and password file <none>
I20260806 16:09:26.382189 25813 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:26.382311 25813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:26.382624 25813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:09:26 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032566, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:09:26.414770 25813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "a1c8e84e03b94908b234cb2e7692c912"
format_stamp: "Formatted at 2026-08-06 16:09:26 on dist-test-slave-7z32"
server_key: "210423556ec013a6e0bd05a16f625903"
server_key_iv: "d4740e99c9d9e9a1bd5fbea6b7d1895c"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:26.415989 25813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "a1c8e84e03b94908b234cb2e7692c912"
format_stamp: "Formatted at 2026-08-06 16:09:26 on dist-test-slave-7z32"
server_key: "210423556ec013a6e0bd05a16f625903"
server_key_iv: "d4740e99c9d9e9a1bd5fbea6b7d1895c"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:26.423885 25813 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260806 16:09:26.467442 25841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:26.469738 25813 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.004s sys 0.000s
I20260806 16:09:26.469973 25813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "a1c8e84e03b94908b234cb2e7692c912"
format_stamp: "Formatted at 2026-08-06 16:09:26 on dist-test-slave-7z32"
server_key: "210423556ec013a6e0bd05a16f625903"
server_key_iv: "d4740e99c9d9e9a1bd5fbea6b7d1895c"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:26.470147 25813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:26.489033 25813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:26.495036 25813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:26.495261 25813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:26.496150 25813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:26.497550 25813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:26.497726 25813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 16:09:26.497893 25813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:26.497998 25813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:26.512835 25813 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:35401
I20260806 16:09:26.516889 26001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:35401 every 8 connection(s)
I20260806 16:09:26.518035 25813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:09:26.522672 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 25813
I20260806 16:09:26.522871 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:09:26.523267 8388 external_mini_cluster.cc:1428] Setting key 0b2e097f44ea398cca972f8b45487329
Aug 06 16:09:26 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032566, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
I20260806 16:09:26.544576 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:26.523133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:47019 (local address 127.8.49.62:42957)
0806 16:09:26.523931 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:26.523937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:26.525417 (+ 1480us) server_negotiation.cc:407] Connection header received
0806 16:09:26.527027 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:26.527031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:26.527093 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:26.527193 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:26.529504 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:26.530638 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:26.533000 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:26.533506 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:26.533627 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:26.538248 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:26.538255 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:26.538260 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:26.538304 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:26.541083 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:26.542013 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:26.542017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:26.542018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:26.542185 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:26.542655 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:26.542664 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:26.542665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:26.542977 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:26.543116 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:26.544086 (+ 970us) server_negotiation.cc:299] Negotiation successful
0806 16:09:26.544266 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":111,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:09:26.548890 26006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:26.521904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42957 (local address 127.8.49.1:47019)
0806 16:09:26.524270 (+ 2366us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:26.524349 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:26.526551 (+ 2202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:26.527588 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:26.527600 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:26.527999 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:26.528884 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:26.528920 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:26.530930 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:26.530936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:26.532782 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:26.532803 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:26.533973 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:26.533983 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:26.534338 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:26.535384 (+ 1046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:26.535418 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:26.537882 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:26.541260 (+ 3378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:26.541270 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:26.541307 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:26.541788 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:26.542313 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:26.542317 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:26.542319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:26.542511 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:26.543108 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:26.543117 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:26.543674 (+ 557us) client_negotiation.cc:769] Sending connection context
0806 16:09:26.543998 (+ 324us) client_negotiation.cc:240] Negotiation successful
0806 16:09:26.544519 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2097,"thread_start_us":208,"threads_started":1}
I20260806 16:09:26.551447 26004 heartbeater.cc:347] Connected to a master server at 127.8.49.62:42957
I20260806 16:09:26.551795 26004 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:26.552657 26004 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
I20260806 16:09:26.555725 25702 ts_manager.cc:194] Registered new tserver with Master: a1c8e84e03b94908b234cb2e7692c912 (127.8.49.1:35401)
I20260806 16:09:26.557654 25702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:47019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:09:26.574934 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:33883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:42957
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:09:26 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786032566, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 16:09:26.771067 26012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:26.771750 26012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:26.771874 26012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:26.780531 26012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:26.780684 26012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:26.780812 26012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:09:26.793455 26012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.49.20:33883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:42957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:26.796571 26012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:26.799249 26012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:26.820477 26018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:26.820516 26022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:26.820484 26019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:26.821925 26012 server_base.cc:1034] running on GCE node
I20260806 16:09:26.822906 26012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:26.824322 26012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:26.825783 26012 hybrid_clock.cc:648] HybridClock initialized: now 1786032566825709 us; error 108 us; skew 500 ppm
Aug 06 16:09:26 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032566, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:26.830699 26012 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:09:26.832226 26012 webserver.cc:460] Webserver started at http://127.8.49.2:37417/ using document root <none> and password file <none>
I20260806 16:09:26.833006 26012 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:26.833112 26012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:26.833405 26012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:09:26 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032566, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:09:26.857993 26012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "60e36ff34cf144fbbac88badfca3e652"
format_stamp: "Formatted at 2026-08-06 16:09:26 on dist-test-slave-7z32"
server_key: "c7e8ef6ba79f02e839dcf915be7ef09a"
server_key_iv: "f1e03453bb5030dd84f19351173d73d4"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:26.859756 26012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "60e36ff34cf144fbbac88badfca3e652"
format_stamp: "Formatted at 2026-08-06 16:09:26 on dist-test-slave-7z32"
server_key: "c7e8ef6ba79f02e839dcf915be7ef09a"
server_key_iv: "f1e03453bb5030dd84f19351173d73d4"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:26.869004 26012 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.005s
I20260806 16:09:26.897877 26031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:26.900377 26012 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.001s
I20260806 16:09:26.900576 26012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "60e36ff34cf144fbbac88badfca3e652"
format_stamp: "Formatted at 2026-08-06 16:09:26 on dist-test-slave-7z32"
server_key: "c7e8ef6ba79f02e839dcf915be7ef09a"
server_key_iv: "f1e03453bb5030dd84f19351173d73d4"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:26.900741 26012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:26.936830 26012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:26.941334 26012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:26.941636 26012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:26.942499 26012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:26.943862 26012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:26.943991 26012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 16:09:26.944085 26012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:26.944134 26012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:26.961055 26012 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:41499
I20260806 16:09:26.963673 26012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:09:26.966593 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 26012
I20260806 16:09:26.966738 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:09:26.967173 8388 external_mini_cluster.cc:1428] Setting key edc2c5418db528c213f6d33f9454dab0
I20260806 16:09:26.965821 26198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:41499 every 8 connection(s)
Aug 06 16:09:26 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032566, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:09:27.006234 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:26.967972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:38809 (local address 127.8.49.62:42957)
0806 16:09:26.968138 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:26.968142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:26.970104 (+ 1962us) server_negotiation.cc:407] Connection header received
0806 16:09:26.975502 (+ 5398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:26.975507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:26.975585 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:26.975650 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:26.981484 (+ 5834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:26.982580 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:26.986867 (+ 4287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:26.987279 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:26.987386 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:26.991503 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:26.991509 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:26.991513 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:26.991552 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:26.993533 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:26.994333 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:26.994339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:26.994342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:26.994465 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:27.004340 (+ 9875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:27.004346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:27.004347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:27.004632 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:27.004867 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:27.005827 (+ 960us) server_negotiation.cc:299] Negotiation successful
0806 16:09:27.006029 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:09:27.007587 26202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:26.968924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42957 (local address 127.8.49.2:38809)
0806 16:09:26.969896 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:26.969978 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:26.975115 (+ 5137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:26.976648 (+ 1533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:26.976661 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:26.977146 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:26.981165 (+ 4019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:26.981202 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:26.982951 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:26.982955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:26.986553 (+ 3598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:26.986583 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:26.987703 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:26.987708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:26.987972 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:26.988849 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:26.988877 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:26.991185 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:26.993737 (+ 2552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:26.993747 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:26.993771 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:26.994194 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:26.994598 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:26.994601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:26.994603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:26.994854 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:27.004864 (+ 10010us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:27.004876 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:27.005306 (+ 430us) client_negotiation.cc:769] Sending connection context
0806 16:09:27.005739 (+ 433us) client_negotiation.cc:240] Negotiation successful
0806 16:09:27.006204 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":317,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:09:27.009423 26199 heartbeater.cc:347] Connected to a master server at 127.8.49.62:42957
I20260806 16:09:27.009747 26199 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:27.010509 26199 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
I20260806 16:09:27.012136 25702 ts_manager.cc:194] Registered new tserver with Master: 60e36ff34cf144fbbac88badfca3e652 (127.8.49.2:41499)
I20260806 16:09:27.012804 25702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:38809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:09:27.056257 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:33883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:42957
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:09:27 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032566, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
W20260806 16:09:27.265555 26206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:27.267023 26206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:27.267409 26206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:27.276328 26206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:27.276737 26206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:27.276938 26206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:09:27.289534 26206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.49.20:33883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:42957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:27.292169 26206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:27.294265 26206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:27.312470 26213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:27.313838 26212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:27.316161 26206 server_base.cc:1034] running on GCE node
W20260806 16:09:27.316215 26215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:27.318408 26206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:27.319438 26206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:27.322799 26206 hybrid_clock.cc:648] HybridClock initialized: now 1786032567322740 us; error 89 us; skew 500 ppm
Aug 06 16:09:27 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032567, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:27.327121 26206 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:09:27.328528 26206 webserver.cc:460] Webserver started at http://127.8.49.3:38771/ using document root <none> and password file <none>
I20260806 16:09:27.329473 26206 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:27.329638 26206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:27.329941 26206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:09:27 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032567, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:09:27.354652 26206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "69e30f2776224cb6a783c34e058836bd"
format_stamp: "Formatted at 2026-08-06 16:09:27 on dist-test-slave-7z32"
server_key: "c00077e7320857150acb1fc6a192a012"
server_key_iv: "89a89dd1b21ca443f5b154682899f057"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:27.357604 26206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "69e30f2776224cb6a783c34e058836bd"
format_stamp: "Formatted at 2026-08-06 16:09:27 on dist-test-slave-7z32"
server_key: "c00077e7320857150acb1fc6a192a012"
server_key_iv: "89a89dd1b21ca443f5b154682899f057"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:27.370338 26206 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.006s sys 0.000s
I20260806 16:09:27.407315 26227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:27.412159 26206 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.005s sys 0.000s
I20260806 16:09:27.412590 26206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "69e30f2776224cb6a783c34e058836bd"
format_stamp: "Formatted at 2026-08-06 16:09:27 on dist-test-slave-7z32"
server_key: "c00077e7320857150acb1fc6a192a012"
server_key_iv: "89a89dd1b21ca443f5b154682899f057"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:27.412904 26206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:27.442484 26206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:27.446583 26206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:27.446933 26206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:27.447837 26206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:27.449075 26206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:27.449204 26206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:27.449312 26206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:27.449383 26206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:27.465106 26206 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:40157
I20260806 16:09:27.466557 26206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:09:27.468734 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 26206
I20260806 16:09:27.468935 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:09:27.469378 8388 external_mini_cluster.cc:1428] Setting key ea2a5dcd18227d3f20e135ec8bb88a38
I20260806 16:09:27.468685 26385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:40157 every 8 connection(s)
Aug 06 16:09:27 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032567, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:27.492583 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:27.475343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:44259 (local address 127.8.49.62:42957)
0806 16:09:27.475468 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:27.475473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:27.475937 (+ 464us) server_negotiation.cc:407] Connection header received
0806 16:09:27.477401 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:27.477405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:27.477466 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:27.477522 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:27.479540 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:27.480396 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:27.482223 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:27.482507 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:27.482591 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:27.486874 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:27.486880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:27.486883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:27.486917 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:27.489263 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:27.490295 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:27.490300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:27.490302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:27.490403 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:27.490970 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:27.490974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:27.490976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:27.491249 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:27.491359 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:27.492225 (+ 866us) server_negotiation.cc:299] Negotiation successful
0806 16:09:27.492382 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 16:09:27.496781 26392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:27.474133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42957 (local address 127.8.49.3:44259)
0806 16:09:27.475768 (+ 1635us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:27.475831 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:27.477130 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:27.478076 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:27.478087 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:27.478542 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:27.479349 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:27.479371 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:27.480576 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:27.480581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:27.482062 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:27.482076 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:27.482816 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:27.482840 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:27.483090 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:27.484574 (+ 1484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:27.484597 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:27.486591 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:27.489682 (+ 3091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:27.489691 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:27.489714 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:27.490154 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:27.490578 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:27.490581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:27.490583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:27.490809 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:27.491451 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:27.491459 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:27.491902 (+ 443us) client_negotiation.cc:769] Sending connection context
0806 16:09:27.493113 (+ 1211us) client_negotiation.cc:240] Negotiation successful
0806 16:09:27.493528 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1389,"thread_start_us":175,"threads_started":1}
I20260806 16:09:27.498641 26386 heartbeater.cc:347] Connected to a master server at 127.8.49.62:42957
I20260806 16:09:27.498989 26386 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:27.499747 26386 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
I20260806 16:09:27.501228 25702 ts_manager.cc:194] Registered new tserver with Master: 69e30f2776224cb6a783c34e058836bd (127.8.49.3:40157)
I20260806 16:09:27.502038 25702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:44259
I20260806 16:09:27.508606 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:09:27.509687 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 25813
I20260806 16:09:27.600731 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 26012
I20260806 16:09:27.620721 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 26206
I20260806 16:09:27.643081 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 25657
I20260806 16:09:27.665339 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42957
--webserver_interface=127.8.49.62
--webserver_port=34103
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:42957
--ranger_config_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:33883/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.49.20:33883/kms with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:27.822662 26393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:27.823148 26393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:27.823254 26393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:27.828480 26393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:09:27.828642 26393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:27.828670 26393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:27.828693 26393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:09:27.828713 26393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:09:27.835929 26393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.49.20:33883/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:42957
--ranger_config_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=34103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:27.837656 26393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:27.839605 26393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:27.852428 26406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:27.852428 26400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:27.852432 26401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:27.853272 26393 server_base.cc:1034] running on GCE node
I20260806 16:09:27.854080 26393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:27.855891 26393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:27.857250 26393 hybrid_clock.cc:648] HybridClock initialized: now 1786032567857201 us; error 56 us; skew 500 ppm
Aug 06 16:09:27 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032567, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:27.861431 26393 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:09:27.863111 26393 webserver.cc:460] Webserver started at http://127.8.49.62:34103/ using document root <none> and password file <none>
I20260806 16:09:27.863926 26393 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:27.864028 26393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:09:27 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032567, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:09:27.956985 26393 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:09:27.960882 26415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:27.962999 26393 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:09:27.963207 26393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "00011490fd234c21a11fa1a23ad72c63"
format_stamp: "Formatted at 2026-08-06 16:09:24 on dist-test-slave-7z32"
server_key: "3352a15c06235dca4f9e050b5c65b917"
server_key_iv: "6a09a6072211522071ac0e144cd4e1d5"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:27.964037 26393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:27.983059 26393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:27.988412 26393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:27.989032 26393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:28.003786 26493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:42957 every 8 connection(s)
I20260806 16:09:28.003813 26393 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:42957
I20260806 16:09:28.006121 26393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:09:28.009755 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 26393
I20260806 16:09:28.010407 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:33883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:35401
--local_ip_for_outbound_sockets=127.8.49.1
--tserver_master_addrs=127.8.49.62:42957
--webserver_port=34879
--webserver_interface=127.8.49.1
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.49.20:33883/kms with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:09:28.013631 26494 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:09:28.016376 26494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63: Bootstrap starting.
I20260806 16:09:28.047976 26494 log.cc:824] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:28.059109 26494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:09:28.059930 26494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63: Bootstrap complete.
I20260806 16:09:28.067039 26494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } }
I20260806 16:09:28.067792 26494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00011490fd234c21a11fa1a23ad72c63, State: Initialized, Role: FOLLOWER
I20260806 16:09:28.068595 26494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } }
I20260806 16:09:28.068828 26494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:09:28.068909 26494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:09:28.069046 26494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:09:28.072999 26494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } }
I20260806 16:09:28.073575 26494 leader_election.cc:302] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00011490fd234c21a11fa1a23ad72c63; no voters:
I20260806 16:09:28.074141 26494 leader_election.cc:288] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 16:09:28.074404 26501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:09:28.075009 26501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [term 2 LEADER]: Becoming Leader. State: Replica: 00011490fd234c21a11fa1a23ad72c63, State: Running, Role: LEADER
I20260806 16:09:28.075495 26501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } }
I20260806 16:09:28.076062 26494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:09:28.077476 26505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00011490fd234c21a11fa1a23ad72c63. Latest consensus state: current_term: 2 leader_uuid: "00011490fd234c21a11fa1a23ad72c63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } } }
I20260806 16:09:28.077543 26503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "00011490fd234c21a11fa1a23ad72c63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00011490fd234c21a11fa1a23ad72c63" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42957 } } }
I20260806 16:09:28.077652 26505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:28.077652 26503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63 [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:28.079216 26494 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 16:09:28.079458 26519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:09:28.087872 26519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:09:28.089370 26519 catalog_manager.cc:1259] Loaded cluster ID: ff53f5c3804542f49e9240c83406dfd2
I20260806 16:09:28.089597 26519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:09:28.094740 26519 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:09:28.095860 26519 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63: Loaded TSK: 0
I20260806 16:09:28.097884 26519 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:09:28.237803 26496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:28.238379 26496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:28.238466 26496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:28.247959 26496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:28.248112 26496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:28.248238 26496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:09:28.259255 26496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.49.20:33883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:35401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=34879
--tserver_master_addrs=127.8.49.62:42957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:28.261750 26496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:28.263986 26496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:28.283176 26555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:28.286330 26496 server_base.cc:1034] running on GCE node
W20260806 16:09:28.286933 26552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:28.287364 26553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:28.288976 26496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:28.290234 26496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:28.292639 26496 hybrid_clock.cc:648] HybridClock initialized: now 1786032568292538 us; error 150 us; skew 500 ppm
Aug 06 16:09:28 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032568, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:28.298278 26496 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:09:28.300266 26496 webserver.cc:460] Webserver started at http://127.8.49.1:34879/ using document root <none> and password file <none>
I20260806 16:09:28.301448 26496 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:28.301702 26496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:09:28 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032568, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:09:28.347860 26496 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.004s
I20260806 16:09:28.353284 26566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:28.355608 26496 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260806 16:09:28.355831 26496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "a1c8e84e03b94908b234cb2e7692c912"
format_stamp: "Formatted at 2026-08-06 16:09:26 on dist-test-slave-7z32"
server_key: "210423556ec013a6e0bd05a16f625903"
server_key_iv: "d4740e99c9d9e9a1bd5fbea6b7d1895c"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:28.356865 26496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:28.397292 26496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:28.404994 26496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:28.405441 26496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:28.406639 26496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:28.408308 26496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:28.408463 26496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 16:09:28.408658 26496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:28.408762 26496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:28.428928 26496 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:35401
I20260806 16:09:28.429116 26718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:35401 every 8 connection(s)
I20260806 16:09:28.430553 26496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:09:28.432549 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 26496
I20260806 16:09:28.433321 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:33883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:41499
--local_ip_for_outbound_sockets=127.8.49.2
--tserver_master_addrs=127.8.49.62:42957
--webserver_port=37417
--webserver_interface=127.8.49.2
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.49.20:33883/kms with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:09:28 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032568, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:28.469514 26726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:28.437768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42957 (local address 127.8.49.1:47183)
0806 16:09:28.443463 (+ 5695us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:28.443607 (+ 144us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:28.445387 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:28.448657 (+ 3270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:28.448672 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:28.449299 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:28.450349 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:28.450384 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:28.453606 (+ 3222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:28.453612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:28.455331 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:28.455352 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:28.456870 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:28.456878 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:28.457220 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:28.458150 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:28.458175 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:28.460426 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:28.464597 (+ 4171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:28.464606 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:28.464633 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:28.465141 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:28.465919 (+ 778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:28.465922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:28.465924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:28.466166 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:28.467417 (+ 1251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:28.467425 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:28.467876 (+ 451us) client_negotiation.cc:769] Sending connection context
0806 16:09:28.468113 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 16:09:28.468494 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5211,"thread_start_us":235,"threads_started":1}
I20260806 16:09:28.470309 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:28.437898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:47183 (local address 127.8.49.62:42957)
0806 16:09:28.446380 (+ 8482us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:28.446425 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:28.446503 (+ 78us) server_negotiation.cc:407] Connection header received
0806 16:09:28.447047 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:28.447070 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:28.447830 (+ 760us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:28.450773 (+ 2943us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:28.451285 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:28.453211 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:28.455582 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:28.456526 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:28.456807 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:28.460879 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:28.460894 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:28.460925 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:28.460987 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:28.464347 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:28.465425 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:28.465436 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:28.465447 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:28.465706 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:28.466357 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:28.466362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:28.466365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:28.467227 (+ 862us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:28.467425 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:28.468396 (+ 971us) server_negotiation.cc:299] Negotiation successful
0806 16:09:28.468889 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8171,"thread_start_us":240,"threads_started":1}
I20260806 16:09:28.474375 26720 heartbeater.cc:347] Connected to a master server at 127.8.49.62:42957
I20260806 16:09:28.474808 26720 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:28.475958 26720 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
W20260806 16:09:28.480202 26720 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:42957 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 16:09:28.480432 26720 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:28.480789 26720 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
I20260806 16:09:28.482944 26720 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:28.483394 26720 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
W20260806 16:09:28.484964 26720 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:09:28.610968 26725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:28.611548 26725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:28.611685 26725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:28.618793 26725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:28.619038 26725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:28.619194 26725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:09:28.630494 26725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.49.20:33883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:41499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=37417
--tserver_master_addrs=127.8.49.62:42957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:28.632609 26725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:28.633981 26725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:28.649036 26735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:28.649242 26732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:28.649813 26733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:28.650978 26725 server_base.cc:1034] running on GCE node
I20260806 16:09:28.651638 26725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:28.652616 26725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:28.656023 26725 hybrid_clock.cc:648] HybridClock initialized: now 1786032568655882 us; error 152 us; skew 500 ppm
Aug 06 16:09:28 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032568, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:28.661765 26725 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:09:28.664067 26725 webserver.cc:460] Webserver started at http://127.8.49.2:37417/ using document root <none> and password file <none>
I20260806 16:09:28.665768 26725 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:28.666072 26725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:09:28 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032568, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:09:28.697471 26725 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:09:28.706571 26745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:28.709075 26725 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.000s
I20260806 16:09:28.709314 26725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "60e36ff34cf144fbbac88badfca3e652"
format_stamp: "Formatted at 2026-08-06 16:09:26 on dist-test-slave-7z32"
server_key: "c7e8ef6ba79f02e839dcf915be7ef09a"
server_key_iv: "f1e03453bb5030dd84f19351173d73d4"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:28.710127 26725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:28.727552 26725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:28.733645 26725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:28.733955 26725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:28.734812 26725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:28.736248 26725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:28.736416 26725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:28.736732 26725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:28.736796 26725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:28.753305 26725 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:41499
I20260806 16:09:28.755477 26725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:09:28.756628 26905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:41499 every 8 connection(s)
I20260806 16:09:28.759055 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 26725
I20260806 16:09:28.759871 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.49.20:33883/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:40157
--local_ip_for_outbound_sockets=127.8.49.3
--tserver_master_addrs=127.8.49.62:42957
--webserver_port=38771
--webserver_interface=127.8.49.3
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.49.20:33883/kms with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:09:28 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032568, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:28.780525 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:28.760301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:52881 (local address 127.8.49.62:42957)
0806 16:09:28.760814 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:28.760821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:28.763384 (+ 2563us) server_negotiation.cc:407] Connection header received
0806 16:09:28.765063 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:28.765067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:28.765166 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:28.765267 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:28.766607 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:28.767350 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:28.768972 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:28.769221 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:28.769351 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:28.773063 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:28.773067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:28.773069 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:28.773110 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:28.775760 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:28.776798 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:28.776803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:28.776805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:28.776928 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:28.777863 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:28.777870 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:28.777871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:28.778147 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:28.778743 (+ 596us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:28.780154 (+ 1411us) server_negotiation.cc:299] Negotiation successful
0806 16:09:28.780334 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356}
I20260806 16:09:28.781602 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:28.761415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42957 (local address 127.8.49.2:52881)
0806 16:09:28.762571 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:28.762889 (+ 318us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:28.764773 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:28.765516 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:28.765525 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:28.765860 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:28.766425 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:28.766439 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:28.767563 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:28.767567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:28.768807 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:28.768820 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:28.769410 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:28.769414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:28.769727 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:28.771143 (+ 1416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:28.771167 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:28.772802 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:28.776140 (+ 3338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:28.776153 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:28.776186 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:28.776646 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:28.777125 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:28.777134 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:28.777137 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:28.777436 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:28.778820 (+ 1384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:28.778927 (+ 107us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:28.779326 (+ 399us) client_negotiation.cc:769] Sending connection context
0806 16:09:28.779906 (+ 580us) client_negotiation.cc:240] Negotiation successful
0806 16:09:28.780343 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":268,"threads_started":1}
I20260806 16:09:28.783533 26906 heartbeater.cc:347] Connected to a master server at 127.8.49.62:42957
I20260806 16:09:28.784096 26906 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:28.785058 26906 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
W20260806 16:09:28.787832 26906 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:42957 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 16:09:28.788023 26906 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:28.788272 26906 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
I20260806 16:09:28.789291 26906 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:28.789527 26906 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
W20260806 16:09:28.790664 26906 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:09:29.012606 26909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:29.013299 26909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:29.013448 26909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:29.021119 26909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:29.021274 26909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:29.021389 26909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:09:29.032810 26909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.49.20:33883/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:40157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=38771
--tserver_master_addrs=127.8.49.62:42957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:29.035455 26909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:29.037621 26909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:29.056890 26920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:29.056892 26918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:29.056981 26917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:29.059317 26909 server_base.cc:1034] running on GCE node
I20260806 16:09:29.059893 26909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:29.060768 26909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:29.064110 26909 hybrid_clock.cc:648] HybridClock initialized: now 1786032569064063 us; error 57 us; skew 500 ppm
Aug 06 16:09:29 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032569, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:29.067828 26909 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:09:29.069084 26909 webserver.cc:460] Webserver started at http://127.8.49.3:38771/ using document root <none> and password file <none>
I20260806 16:09:29.069880 26909 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:29.070011 26909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:09:29 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032569, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:09:29.101845 26909 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:09:29.106185 26931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:29.108349 26909 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 16:09:29.108582 26909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "69e30f2776224cb6a783c34e058836bd"
format_stamp: "Formatted at 2026-08-06 16:09:27 on dist-test-slave-7z32"
server_key: "c00077e7320857150acb1fc6a192a012"
server_key_iv: "89a89dd1b21ca443f5b154682899f057"
server_key_version: "kuduclusterkey@0"
I20260806 16:09:29.109375 26909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:29.129480 26909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:29.134290 26909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:29.134543 26909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:29.135526 26909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:29.136857 26909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:29.136929 26909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:29.136994 26909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:29.137096 26909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 16:09:29.152150 26909 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:40157
I20260806 16:09:29.152259 27104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:40157 every 8 connection(s)
I20260806 16:09:29.154031 26909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:09:29.158684 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 26909
Aug 06 16:09:29 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032569, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:29.173924 27115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.159722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53660 (local address 127.8.49.62:42957)
0806 16:09:29.160117 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:29.160129 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:29.162097 (+ 1968us) server_negotiation.cc:407] Connection header received
0806 16:09:29.162274 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:29.162277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:29.162336 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:29.162449 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:29.163652 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.164381 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.165817 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.166013 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.166118 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.168707 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:29.168710 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:29.168712 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:29.168741 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:29.171582 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:29.172093 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:29.172097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:29.172099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:29.172195 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:29.172497 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:29.172503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:29.172506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:29.172846 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:29.173119 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:29.173497 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 16:09:29.173688 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":293,"thread_start_us":145,"threads_started":1}
I20260806 16:09:29.174351 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.157127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:40329 (local address 127.8.49.62:42957)
0806 16:09:29.157350 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:29.157356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:29.158500 (+ 1144us) server_negotiation.cc:407] Connection header received
0806 16:09:29.160069 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:29.160074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:29.160145 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:29.160199 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:29.161850 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.162868 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.165266 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.165621 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.165710 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.169362 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:29.169367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:29.169373 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:29.169437 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:29.171006 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:29.171907 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:29.171913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:29.171916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:29.172066 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:29.172632 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:29.172635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:29.172638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:29.172846 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:29.172953 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:29.173830 (+ 877us) server_negotiation.cc:299] Negotiation successful
0806 16:09:29.174112 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":83,"spinlock_wait_cycles":1408}
I20260806 16:09:29.176518 27111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.157594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42957 (local address 127.8.49.3:40329)
0806 16:09:29.158284 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:29.158363 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:29.159751 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:29.160636 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:29.160644 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:29.160998 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:29.161640 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.161655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.163173 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.163179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.165074 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.165100 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.165824 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.165829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.166082 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.167004 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:29.167035 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:29.169102 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:29.171202 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:29.171210 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:29.171260 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:29.171728 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:29.172216 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:29.172219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:29.172222 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:29.172449 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:29.172987 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:29.172995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:29.173444 (+ 449us) client_negotiation.cc:769] Sending connection context
0806 16:09:29.174579 (+ 1135us) client_negotiation.cc:240] Negotiation successful
0806 16:09:29.175096 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":251,"threads_started":1}
I20260806 16:09:29.178743 27105 heartbeater.cc:347] Connected to a master server at 127.8.49.62:42957
I20260806 16:09:29.179314 27105 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:29.180330 27105 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
I20260806 16:09:29.183454 26436 ts_manager.cc:194] Registered new tserver with Master: 69e30f2776224cb6a783c34e058836bd (127.8.49.3:40157)
I20260806 16:09:29.185545 26436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:40329
I20260806 16:09:29.485350 26720 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:29.485809 26720 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
I20260806 16:09:29.487087 26436 ts_manager.cc:194] Registered new tserver with Master: a1c8e84e03b94908b234cb2e7692c912 (127.8.49.1:35401)
I20260806 16:09:29.488010 26436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:47183
Aug 06 16:09:29 dist-test-slave-7z32 krb5kdc[25034](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786032569, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:29.791013 26906 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:29.791635 26906 heartbeater.cc:511] Master 127.8.49.62:42957 requested a full tablet report, sending...
I20260806 16:09:29.793200 26436 ts_manager.cc:194] Registered new tserver with Master: 60e36ff34cf144fbbac88badfca3e652 (127.8.49.2:41499)
I20260806 16:09:29.794399 26436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:52881
I20260806 16:09:29.802289 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:09:29.826516 27133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.805403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53668 (local address 127.8.49.62:42957)
0806 16:09:29.805856 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:29.805931 (+ 75us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:29.806348 (+ 417us) server_negotiation.cc:407] Connection header received
0806 16:09:29.806471 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:29.806476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:29.806564 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:29.806750 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:29.808122 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.809170 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.810710 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.811347 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.811794 (+ 447us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.820507 (+ 8713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:29.820514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:29.820518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:29.820581 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:29.823259 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:29.824061 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:29.824065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:29.824067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:29.824187 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:29.824501 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:29.824504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:29.824506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:29.824803 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:29.824944 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:29.825891 (+ 947us) server_negotiation.cc:299] Negotiation successful
0806 16:09:29.826055 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":163,"threads_started":1}
I20260806 16:09:29.834227 26436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:09:29.841899 26436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:09:29.874121 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.864116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:40157 (local address 127.0.0.1:51554)
0806 16:09:29.866742 (+ 2626us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:29.866757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:29.866989 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:29.868106 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:29.868115 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:29.868148 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:29.868363 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.868373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.869842 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.869848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.871497 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.871511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.872938 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.872942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.873325 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.873400 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 16:09:29.873664 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 16:09:29.873839 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2554,"mutex_wait_us":46,"thread_start_us":174,"threads_started":1}
I20260806 16:09:29.874712 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.864582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.8.49.3:40157)
0806 16:09:29.866566 (+ 1984us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:29.866577 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:29.866902 (+ 325us) server_negotiation.cc:407] Connection header received
0806 16:09:29.867238 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:29.867247 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:29.867674 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:29.867868 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:29.868527 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.869676 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.871656 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.872536 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.872651 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.872852 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:29.874414 (+ 1562us) server_negotiation.cc:299] Negotiation successful
0806 16:09:29.874558 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1833,"thread_start_us":122,"threads_started":1}
I20260806 16:09:29.876624 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.862279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:41499 (local address 127.0.0.1:48656)
0806 16:09:29.865867 (+ 3588us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:29.865958 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:29.866161 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:29.868227 (+ 2066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:29.868230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:29.868253 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:29.868521 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.868531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.870594 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.870600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.871891 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.871902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.873152 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.873156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.873290 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.873487 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 16:09:29.873924 (+ 437us) client_negotiation.cc:240] Negotiation successful
0806 16:09:29.874075 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3299,"mutex_wait_us":223,"thread_start_us":213,"threads_started":1}
I20260806 16:09:29.876976 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.865704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:35401 (local address 127.0.0.1:57448)
0806 16:09:29.867213 (+ 1509us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:29.867226 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:29.867304 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:29.871563 (+ 4259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:29.871567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:29.871589 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:29.871690 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.871696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.873161 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.873164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.874943 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.874959 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.876331 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.876338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.876473 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.876511 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 16:09:29.876627 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 16:09:29.876729 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1444,"thread_start_us":129,"threads_started":1}
I20260806 16:09:29.877288 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.862504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48656 (local address 127.8.49.2:41499)
0806 16:09:29.867617 (+ 5113us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:29.867626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:29.867653 (+ 27us) server_negotiation.cc:407] Connection header received
0806 16:09:29.867759 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:29.867765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:29.868034 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:29.868726 (+ 692us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:29.868920 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.870300 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.872038 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.872965 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.876587 (+ 3622us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.876839 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:29.876917 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 16:09:29.877076 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4968,"thread_start_us":163,"threads_started":1}
I20260806 16:09:29.877702 27148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.869981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.8.49.1:35401)
0806 16:09:29.870608 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:29.870617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:29.870645 (+ 28us) server_negotiation.cc:407] Connection header received
0806 16:09:29.870773 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:29.870780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:29.871124 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:29.871323 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:29.871835 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.873022 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.875127 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.876125 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.876283 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.876496 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:29.877390 (+ 894us) server_negotiation.cc:299] Negotiation successful
0806 16:09:29.877560 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":122,"threads_started":1}
I20260806 16:09:29.879498 27004 tablet_service.cc:1467] Processing CreateTablet for tablet ab4a4e8c90b148b6a32e8e36326a9e29 (DEFAULT_TABLE table=test-table [id=9e22f60cee0d41d5b7e02576daaac47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:09:29.880743 27004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab4a4e8c90b148b6a32e8e36326a9e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:29.882767 26805 tablet_service.cc:1467] Processing CreateTablet for tablet ab4a4e8c90b148b6a32e8e36326a9e29 (DEFAULT_TABLE table=test-table [id=9e22f60cee0d41d5b7e02576daaac47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:09:29.884624 26805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab4a4e8c90b148b6a32e8e36326a9e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:29.890774 26614 tablet_service.cc:1467] Processing CreateTablet for tablet ab4a4e8c90b148b6a32e8e36326a9e29 (DEFAULT_TABLE table=test-table [id=9e22f60cee0d41d5b7e02576daaac47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:09:29.893118 26614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab4a4e8c90b148b6a32e8e36326a9e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:29.893285 27149 tablet_bootstrap.cc:492] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd: Bootstrap starting.
I20260806 16:09:29.897468 27149 tablet_bootstrap.cc:654] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:29.898715 27149 log.cc:824] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:29.898793 27150 tablet_bootstrap.cc:492] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: Bootstrap starting.
I20260806 16:09:29.901749 27149 tablet_bootstrap.cc:492] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd: No bootstrap required, opened a new log
I20260806 16:09:29.902019 27149 ts_tablet_manager.cc:1397] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 16:09:29.903069 27150 tablet_bootstrap.cc:654] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:29.904397 27150 log.cc:824] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:29.908098 27149 raft_consensus.cc:348] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e36ff34cf144fbbac88badfca3e652" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41499 } } peers { permanent_uuid: "69e30f2776224cb6a783c34e058836bd" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40157 } } peers { permanent_uuid: "a1c8e84e03b94908b234cb2e7692c912" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35401 } }
I20260806 16:09:29.908425 27149 raft_consensus.cc:374] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:29.908628 27149 raft_consensus.cc:729] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e30f2776224cb6a783c34e058836bd, State: Initialized, Role: FOLLOWER
I20260806 16:09:29.908854 27150 tablet_bootstrap.cc:492] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: No bootstrap required, opened a new log
I20260806 16:09:29.909281 27150 ts_tablet_manager.cc:1397] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20260806 16:09:29.909245 27149 consensus_queue.cc:260] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e36ff34cf144fbbac88badfca3e652" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41499 } } peers { permanent_uuid: "69e30f2776224cb6a783c34e058836bd" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40157 } } peers { permanent_uuid: "a1c8e84e03b94908b234cb2e7692c912" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35401 } }
I20260806 16:09:29.910246 27105 heartbeater.cc:503] Master 127.8.49.62:42957 was elected leader, sending a full tablet report...
I20260806 16:09:29.910461 27149 ts_tablet_manager.cc:1428] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:09:29.911125 27153 tablet_bootstrap.cc:492] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912: Bootstrap starting.
I20260806 16:09:29.914673 27153 tablet_bootstrap.cc:654] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:29.916604 27153 log.cc:824] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:29.917455 27150 raft_consensus.cc:348] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e36ff34cf144fbbac88badfca3e652" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41499 } } peers { permanent_uuid: "69e30f2776224cb6a783c34e058836bd" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40157 } } peers { permanent_uuid: "a1c8e84e03b94908b234cb2e7692c912" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35401 } }
I20260806 16:09:29.917814 27150 raft_consensus.cc:374] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:29.917884 27150 raft_consensus.cc:729] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60e36ff34cf144fbbac88badfca3e652, State: Initialized, Role: FOLLOWER
I20260806 16:09:29.918745 27150 consensus_queue.cc:260] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e36ff34cf144fbbac88badfca3e652" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41499 } } peers { permanent_uuid: "69e30f2776224cb6a783c34e058836bd" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40157 } } peers { permanent_uuid: "a1c8e84e03b94908b234cb2e7692c912" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35401 } }
I20260806 16:09:29.920204 27150 ts_tablet_manager.cc:1428] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: Time spent starting tablet: real 0.011s user 0.006s sys 0.002s
I20260806 16:09:29.920779 26906 heartbeater.cc:503] Master 127.8.49.62:42957 was elected leader, sending a full tablet report...
I20260806 16:09:29.922267 27153 tablet_bootstrap.cc:492] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912: No bootstrap required, opened a new log
I20260806 16:09:29.922708 27153 ts_tablet_manager.cc:1397] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260806 16:09:29.940992 27153 raft_consensus.cc:348] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e36ff34cf144fbbac88badfca3e652" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41499 } } peers { permanent_uuid: "69e30f2776224cb6a783c34e058836bd" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40157 } } peers { permanent_uuid: "a1c8e84e03b94908b234cb2e7692c912" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35401 } }
I20260806 16:09:29.941694 27153 raft_consensus.cc:374] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:29.941828 27153 raft_consensus.cc:729] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1c8e84e03b94908b234cb2e7692c912, State: Initialized, Role: FOLLOWER
I20260806 16:09:29.942704 27153 consensus_queue.cc:260] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e36ff34cf144fbbac88badfca3e652" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41499 } } peers { permanent_uuid: "69e30f2776224cb6a783c34e058836bd" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40157 } } peers { permanent_uuid: "a1c8e84e03b94908b234cb2e7692c912" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35401 } }
I20260806 16:09:29.944206 26720 heartbeater.cc:503] Master 127.8.49.62:42957 was elected leader, sending a full tablet report...
I20260806 16:09:29.944844 27153 ts_tablet_manager.cc:1428] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912: Time spent starting tablet: real 0.013s user 0.007s sys 0.003s
I20260806 16:09:29.945434 27156 raft_consensus.cc:482] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:09:29.945673 27156 raft_consensus.cc:504] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e36ff34cf144fbbac88badfca3e652" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41499 } } peers { permanent_uuid: "69e30f2776224cb6a783c34e058836bd" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40157 } } peers { permanent_uuid: "a1c8e84e03b94908b234cb2e7692c912" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35401 } }
I20260806 16:09:29.947443 27156 leader_election.cc:288] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69e30f2776224cb6a783c34e058836bd (127.8.49.3:40157), a1c8e84e03b94908b234cb2e7692c912 (127.8.49.1:35401)
I20260806 16:09:29.956472 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.949060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:52753 (local address 127.8.49.3:40157)
0806 16:09:29.949238 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:29.949244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:29.951258 (+ 2014us) server_negotiation.cc:407] Connection header received
0806 16:09:29.951429 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:29.951433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:29.951505 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:29.951606 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:29.952258 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.953132 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.954987 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.955632 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.955725 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.955808 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:29.956218 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 16:09:29.956311 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:09:29.957261 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.948002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:40157 (local address 127.8.49.2:52753)
0806 16:09:29.951051 (+ 3049us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:29.951072 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:29.951290 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:29.951783 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:29.951788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:29.951898 (+ 110us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:29.952091 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.952099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.953307 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.953310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.954758 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.954773 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.955926 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.955930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.956057 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.956087 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 16:09:29.957006 (+ 919us) client_negotiation.cc:240] Negotiation successful
0806 16:09:29.957109 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2944,"mutex_wait_us":15,"spinlock_wait_cycles":27904,"thread_start_us":210,"threads_started":1}
I20260806 16:09:29.958052 27035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4a4e8c90b148b6a32e8e36326a9e29" candidate_uuid: "60e36ff34cf144fbbac88badfca3e652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e30f2776224cb6a783c34e058836bd" is_pre_election: true
I20260806 16:09:29.958441 27035 raft_consensus.cc:2454] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60e36ff34cf144fbbac88badfca3e652 in term 0.
I20260806 16:09:29.959100 26747 leader_election.cc:302] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60e36ff34cf144fbbac88badfca3e652, 69e30f2776224cb6a783c34e058836bd; no voters:
I20260806 16:09:29.959192 27148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.949202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:40641 (local address 127.8.49.1:35401)
0806 16:09:29.949342 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:29.949347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:29.951734 (+ 2387us) server_negotiation.cc:407] Connection header received
0806 16:09:29.951997 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:29.952001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:29.952069 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:29.952125 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:29.952724 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.953520 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.957702 (+ 4182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.958323 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.958445 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.958533 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:29.958975 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 16:09:29.959062 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:09:29.959585 27161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.948602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:35401 (local address 127.8.49.2:40641)
0806 16:09:29.951613 (+ 3011us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:29.951630 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:29.951853 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:29.952321 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:29.952324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:29.952395 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:29.952570 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.952578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.953696 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.953700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.954995 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:29.955009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.958648 (+ 3639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.958651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:29.958777 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.958846 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 16:09:29.959326 (+ 480us) client_negotiation.cc:240] Negotiation successful
0806 16:09:29.959416 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2932,"thread_start_us":116,"threads_started":1}
I20260806 16:09:29.959535 27156 raft_consensus.cc:2781] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:09:29.959805 27156 raft_consensus.cc:482] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:09:29.959873 27156 raft_consensus.cc:3037] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:29.960749 26631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4a4e8c90b148b6a32e8e36326a9e29" candidate_uuid: "60e36ff34cf144fbbac88badfca3e652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1c8e84e03b94908b234cb2e7692c912" is_pre_election: true
I20260806 16:09:29.961093 26631 raft_consensus.cc:2454] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60e36ff34cf144fbbac88badfca3e652 in term 0.
I20260806 16:09:29.961504 27156 raft_consensus.cc:504] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e36ff34cf144fbbac88badfca3e652" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41499 } } peers { permanent_uuid: "69e30f2776224cb6a783c34e058836bd" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40157 } } peers { permanent_uuid: "a1c8e84e03b94908b234cb2e7692c912" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35401 } }
I20260806 16:09:29.961907 27156 leader_election.cc:288] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [CANDIDATE]: Term 1 election: Requested vote from peers 69e30f2776224cb6a783c34e058836bd (127.8.49.3:40157), a1c8e84e03b94908b234cb2e7692c912 (127.8.49.1:35401)
I20260806 16:09:29.962335 27035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4a4e8c90b148b6a32e8e36326a9e29" candidate_uuid: "60e36ff34cf144fbbac88badfca3e652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e30f2776224cb6a783c34e058836bd"
I20260806 16:09:29.962338 26631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab4a4e8c90b148b6a32e8e36326a9e29" candidate_uuid: "60e36ff34cf144fbbac88badfca3e652" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1c8e84e03b94908b234cb2e7692c912"
I20260806 16:09:29.962463 26631 raft_consensus.cc:3037] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:29.962478 27035 raft_consensus.cc:3037] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:29.963923 27035 raft_consensus.cc:2454] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60e36ff34cf144fbbac88badfca3e652 in term 1.
I20260806 16:09:29.963923 26631 raft_consensus.cc:2454] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60e36ff34cf144fbbac88badfca3e652 in term 1.
I20260806 16:09:29.964370 26747 leader_election.cc:302] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60e36ff34cf144fbbac88badfca3e652, 69e30f2776224cb6a783c34e058836bd; no voters:
I20260806 16:09:29.964643 27156 raft_consensus.cc:2781] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:09:29.964942 27156 raft_consensus.cc:686] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 1 LEADER]: Becoming Leader. State: Replica: 60e36ff34cf144fbbac88badfca3e652, State: Running, Role: LEADER
I20260806 16:09:29.965351 27156 consensus_queue.cc:237] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e36ff34cf144fbbac88badfca3e652" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41499 } } peers { permanent_uuid: "69e30f2776224cb6a783c34e058836bd" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40157 } } peers { permanent_uuid: "a1c8e84e03b94908b234cb2e7692c912" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35401 } }
I20260806 16:09:29.969378 26436 catalog_manager.cc:5495] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60e36ff34cf144fbbac88badfca3e652 (127.8.49.2). New cstate: current_term: 1 leader_uuid: "60e36ff34cf144fbbac88badfca3e652" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60e36ff34cf144fbbac88badfca3e652" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 41499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69e30f2776224cb6a783c34e058836bd" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1c8e84e03b94908b234cb2e7692c912" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35401 } health_report { overall_health: UNKNOWN } } }
I20260806 16:09:29.990141 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:29.982269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48658 (local address 127.8.49.2:41499)
0806 16:09:29.982570 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:29.982576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:29.982773 (+ 197us) server_negotiation.cc:407] Connection header received
0806 16:09:29.983218 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:29.983253 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:29.983405 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:29.983587 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:09:29.984237 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.985198 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.986955 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:29.987331 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:29.987462 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:29.988102 (+ 640us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:09:29.989039 (+ 937us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:09:29.989273 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:29.989830 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 16:09:29.989933 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260806 16:09:30.002235 26631 raft_consensus.cc:1261] T ab4a4e8c90b148b6a32e8e36326a9e29 P a1c8e84e03b94908b234cb2e7692c912 [term 1 FOLLOWER]: Refusing update from remote peer 60e36ff34cf144fbbac88badfca3e652: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:09:30.003790 27035 raft_consensus.cc:1261] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd [term 1 FOLLOWER]: Refusing update from remote peer 60e36ff34cf144fbbac88badfca3e652: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:09:30.005355 27162 consensus_queue.cc:1035] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1c8e84e03b94908b234cb2e7692c912" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:09:30.006008 27162 consensus_queue.cc:1035] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69e30f2776224cb6a783c34e058836bd" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 16:09:30.008816 26907 tablet.cc:2369] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:09:30.039662 27170 mvcc.cc:205] Tried to move back new op lower bound from 7315589406714699776 to 7315589406581407744. Current Snapshot: MvccSnapshot[applied={T|T < 7315589406714699776}]
I20260806 16:09:30.068419 26436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:09:30.069110 26436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:09:30.073482 26436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00011490fd234c21a11fa1a23ad72c63: Sending DeleteTablet for 3 replicas of tablet ab4a4e8c90b148b6a32e8e36326a9e29
I20260806 16:09:30.075026 27004 tablet_service.cc:1514] Processing DeleteTablet for tablet ab4a4e8c90b148b6a32e8e36326a9e29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:09:30 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:51554
I20260806 16:09:30.075301 26614 tablet_service.cc:1514] Processing DeleteTablet for tablet ab4a4e8c90b148b6a32e8e36326a9e29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:09:30 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:57448
I20260806 16:09:30.075718 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 26496
I20260806 16:09:30.083222 27178 tablet_replica.cc:330] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd: stopping tablet replica
I20260806 16:09:30.083971 27178 raft_consensus.cc:2229] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:09:30.084498 26805 tablet_service.cc:1514] Processing DeleteTablet for tablet ab4a4e8c90b148b6a32e8e36326a9e29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:09:30 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:48656
I20260806 16:09:30.084743 27178 raft_consensus.cc:2258] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:09:30.087781 27178 ts_tablet_manager.cc:1905] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:09:30.092679 27181 tablet_replica.cc:330] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: stopping tablet replica
I20260806 16:09:30.093398 27181 raft_consensus.cc:2229] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:09:30.097162 27178 ts_tablet_manager.cc:1918] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:09:30.097958 27181 raft_consensus.cc:2258] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:09:30.104781 27181 ts_tablet_manager.cc:1905] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:09:30.107206 27178 log.cc:1196] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/ab4a4e8c90b148b6a32e8e36326a9e29
I20260806 16:09:30.107757 27178 ts_tablet_manager.cc:1939] T ab4a4e8c90b148b6a32e8e36326a9e29 P 69e30f2776224cb6a783c34e058836bd: Deleting consensus metadata
I20260806 16:09:30.109606 26418 catalog_manager.cc:4841] TS 69e30f2776224cb6a783c34e058836bd (127.8.49.3:40157): tablet ab4a4e8c90b148b6a32e8e36326a9e29 (table test-table [id=9e22f60cee0d41d5b7e02576daaac47f]) successfully deleted
I20260806 16:09:30.111523 27181 ts_tablet_manager.cc:1918] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:09:30.111850 27181 log.cc:1196] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/ab4a4e8c90b148b6a32e8e36326a9e29
I20260806 16:09:30.112718 27181 ts_tablet_manager.cc:1939] T ab4a4e8c90b148b6a32e8e36326a9e29 P 60e36ff34cf144fbbac88badfca3e652: Deleting consensus metadata
W20260806 16:09:30.114782 26419 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:35401 (error 108)
I20260806 16:09:30.115829 26417 catalog_manager.cc:4841] TS 60e36ff34cf144fbbac88badfca3e652 (127.8.49.2:41499): tablet ab4a4e8c90b148b6a32e8e36326a9e29 (table test-table [id=9e22f60cee0d41d5b7e02576daaac47f]) successfully deleted
I20260806 16:09:30.116421 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 26725
I20260806 16:09:30.128214 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:30.127712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:35401 (local address 127.0.0.1:57464)
0806 16:09:30.128000 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:30.128098 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:35401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
W20260806 16:09:30.128485 26419 catalog_manager.cc:4584] TS a1c8e84e03b94908b234cb2e7692c912 (127.8.49.1:35401): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab4a4e8c90b148b6a32e8e36326a9e29: Network error: Client connection negotiation failed: client connection to 127.8.49.1:35401: connect: Connection refused (error 111)
I20260806 16:09:30.139564 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 26909
Aug 06 16:09:30 dist-test-slave-7z32 krb5kdc[25034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032569, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for HTTP/127.8.49.20@KRBTEST.COM
I20260806 16:09:30.182672 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:30.182174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:35401 (local address 127.0.0.1:57466)
0806 16:09:30.182438 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:30.182550 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:35401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 16:09:30.183424 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 26393
I20260806 16:09:30.206250 8388 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 16:09:30.563164 8388 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 16:09:30.563316 8388 mini_ranger.cc:66] Stopping Ranger...
I20260806 16:09:30.931272 8388 mini_ranger.cc:68] Stopped Ranger
2026-08-06 16:09:30.931 UTC [25074] LOG: received smart shutdown request
2026-08-06 16:09:30.937 UTC [25074] LOG: background worker "logical replication launcher" (PID 25088) exited with exit code 1
2026-08-06 16:09:30.938 UTC [25079] LOG: shutting down
2026-08-06 16:09:31.177 UTC [25074] LOG: database system is shut down
2026-08-06T16:09:31Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (64141 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:09:31 dist-test-slave-7z32 krb5kdc[27195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:09:31 dist-test-slave-7z32 krb5kdc[27195](info): set up 2 sockets
Aug 06 16:09:31 dist-test-slave-7z32 krb5kdc[27195](info): commencing operation
krb5kdc: starting...
W20260806 16:09:32.717780 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.359s user 0.001s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:09:32 dist-test-slave-7z32 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:09:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:09:32.883543 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:38001
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:38001,127.8.49.61:34179,127.8.49.60:34785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:33.030364 27217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:33.030733 27217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:33.030779 27217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:33.035218 27217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:09:33.035312 27217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:33.035334 27217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:33.035351 27217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:09:33.035368 27217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:09:33.042498 27217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:38001,127.8.49.61:34179,127.8.49.60:34785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:38001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:33.044103 27217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:33.045663 27217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:33.056972 27227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:33.056998 27223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:33.056973 27230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:33.057502 27217 server_base.cc:1034] running on GCE node
I20260806 16:09:33.058218 27217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:33.059855 27217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:33.061079 27217 hybrid_clock.cc:648] HybridClock initialized: now 1786032573061055 us; error 53 us; skew 500 ppm
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:33.064670 27217 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:09:33.065901 27217 webserver.cc:460] Webserver started at http://127.8.49.62:46201/ using document root <none> and password file <none>
I20260806 16:09:33.066673 27217 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:33.066771 27217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:33.067070 27217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:09:33.069576 27217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "310cc0dd23ce43a4b739f3938bbd831a"
format_stamp: "Formatted at 2026-08-06 16:09:33 on dist-test-slave-7z32"
server_key: "380a5f6c42c7c9a9aa552dec74939f62"
server_key_iv: "827fc42e77e070f9193569ac3ad27c68"
server_key_version: "encryptionkey@0"
I20260806 16:09:33.070241 27217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "310cc0dd23ce43a4b739f3938bbd831a"
format_stamp: "Formatted at 2026-08-06 16:09:33 on dist-test-slave-7z32"
server_key: "380a5f6c42c7c9a9aa552dec74939f62"
server_key_iv: "827fc42e77e070f9193569ac3ad27c68"
server_key_version: "encryptionkey@0"
I20260806 16:09:33.075212 27217 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:09:33.078550 27238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:33.080273 27217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:09:33.080461 27217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "310cc0dd23ce43a4b739f3938bbd831a"
format_stamp: "Formatted at 2026-08-06 16:09:33 on dist-test-slave-7z32"
server_key: "380a5f6c42c7c9a9aa552dec74939f62"
server_key_iv: "827fc42e77e070f9193569ac3ad27c68"
server_key_version: "encryptionkey@0"
I20260806 16:09:33.080603 27217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:33.103571 27217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:33.108593 27217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:33.108899 27217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:33.119619 27217 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:38001
I20260806 16:09:33.119665 27322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:38001 every 8 connection(s)
I20260806 16:09:33.120968 27217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:09:33.124751 27323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:33.125425 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 27217
I20260806 16:09:33.125574 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:09:33.125946 8388 external_mini_cluster.cc:1428] Setting key 1220754668ede383807f07c65eb9b548
I20260806 16:09:33.130482 27323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.49.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.61@KRBTEST.COM" created.
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:33.151315 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.133831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:38001 (local address 127.0.0.1:59634)
0806 16:09:33.134553 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.134608 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:33.135719 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:33.136488 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:33.136494 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:33.137014 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:33.137325 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.137345 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.138712 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.138716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.139689 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.139701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.140450 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.140454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.140598 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.143054 (+ 2456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:33.143081 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:33.145490 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:33.147843 (+ 2353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.147854 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.147867 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.148120 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.148481 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.148484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.148486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.148612 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.149300 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:33.149307 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:33.149478 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 16:09:33.149671 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 16:09:33.150154 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":154,"thread_start_us":247,"threads_started":1}
I20260806 16:09:33.151325 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.133825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59634 (local address 127.8.49.62:38001)
0806 16:09:33.134396 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.134420 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.134744 (+ 324us) server_negotiation.cc:407] Connection header received
0806 16:09:33.136076 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.136081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.136275 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.136488 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.137511 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.138577 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.139813 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.140198 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.140473 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.145697 (+ 5224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.145707 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.145727 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.145749 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.147706 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.148260 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.148266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.148267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.148378 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.148712 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.148717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.148723 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.149173 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.149337 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.149757 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.150118 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":378,"thread_start_us":194,"threads_started":1}
I20260806 16:09:33.154963 27323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:09:33.156268 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.155648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:34179 (local address 127.0.0.1:43070)
0806 16:09:33.155899 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.156108 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260806 16:09:33.156675 27242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
I20260806 16:09:33.158715 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.158203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:34179 (local address 127.0.0.1:43086)
0806 16:09:33.158428 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.158601 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 16:09:33.158943 27323 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:34179: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab.
Entry for principal kudu/127.8.49.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab.
I20260806 16:09:33.163369 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.61:34179
--webserver_interface=127.8.49.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab
--principal=kudu/127.8.49.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:38001,127.8.49.61:34179,127.8.49.60:34785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:09:33.208300 27323 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } attempt: 1
I20260806 16:09:33.209720 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.209202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:34179 (local address 127.0.0.1:43100)
0806 16:09:33.209513 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.209609 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260806 16:09:33.210927 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.210470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:34179 (local address 127.0.0.1:43102)
0806 16:09:33.210683 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.210781 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 16:09:33.211143 27323 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:34179: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
I20260806 16:09:33.279346 27323 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } attempt: 2
I20260806 16:09:33.280748 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.280165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:34179 (local address 127.0.0.1:43106)
0806 16:09:33.280521 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.280621 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 16:09:33.281811 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.281525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:34179 (local address 127.0.0.1:43118)
0806 16:09:33.281669 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.281737 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 16:09:33.282027 27323 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:34179: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
W20260806 16:09:33.316622 27330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:33.317049 27330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:33.317099 27330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:33.321677 27330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:09:33.321794 27330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:33.321820 27330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:33.321841 27330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:09:33.321857 27330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:09:33.328079 27330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:38001,127.8.49.61:34179,127.8.49.60:34785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab
--principal=kudu/127.8.49.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.61:34179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.49.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:33.329672 27330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:33.331121 27330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:33.342576 27337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:33.342597 27342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:33.342617 27339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:33.343119 27330 server_base.cc:1034] running on GCE node
I20260806 16:09:33.344187 27330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:33.345750 27330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:33.346928 27330 hybrid_clock.cc:648] HybridClock initialized: now 1786032573346907 us; error 43 us; skew 500 ppm
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:33.350903 27330 init.cc:377] Logged in from keytab as kudu/127.8.49.61@KRBTEST.COM (short username kudu)
I20260806 16:09:33.352102 27330 webserver.cc:460] Webserver started at http://127.8.49.61:34895/ using document root <none> and password file <none>
I20260806 16:09:33.352807 27330 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:33.352880 27330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:33.353099 27330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:09:33.356040 27330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/data/instance:
uuid: "84174475343a427799b0c1a06c0bf2de"
format_stamp: "Formatted at 2026-08-06 16:09:33 on dist-test-slave-7z32"
server_key: "83fd6e7107827b4bbf06da3e6befd1d7"
server_key_iv: "abb8037c7d8ecc747305791b7297097d"
server_key_version: "encryptionkey@0"
I20260806 16:09:33.356791 27330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/wal/instance:
uuid: "84174475343a427799b0c1a06c0bf2de"
format_stamp: "Formatted at 2026-08-06 16:09:33 on dist-test-slave-7z32"
server_key: "83fd6e7107827b4bbf06da3e6befd1d7"
server_key_iv: "abb8037c7d8ecc747305791b7297097d"
server_key_version: "encryptionkey@0"
I20260806 16:09:33.362509 27330 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:09:33.366706 27352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:33.369071 27330 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:09:33.369283 27330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/wal
uuid: "84174475343a427799b0c1a06c0bf2de"
format_stamp: "Formatted at 2026-08-06 16:09:33 on dist-test-slave-7z32"
server_key: "83fd6e7107827b4bbf06da3e6befd1d7"
server_key_iv: "abb8037c7d8ecc747305791b7297097d"
server_key_version: "encryptionkey@0"
I20260806 16:09:33.369553 27330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:33.373229 27323 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } attempt: 3
I20260806 16:09:33.374333 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.373937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:34179 (local address 127.0.0.1:43122)
0806 16:09:33.374189 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.374247 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260806 16:09:33.375119 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.374919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:34179 (local address 127.0.0.1:43132)
0806 16:09:33.375011 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.375064 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 16:09:33.375265 27323 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:34179: Network error: Client connection negotiation failed: client connection to 127.8.49.61:34179: connect: Connection refused (error 111)
I20260806 16:09:33.389108 27330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:33.394556 27330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:33.394964 27330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:33.405674 27330 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.61:34179
I20260806 16:09:33.405704 27424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.61:34179 every 8 connection(s)
I20260806 16:09:33.407291 27330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
I20260806 16:09:33.411281 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:33.415199 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 27330
I20260806 16:09:33.415375 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-1/wal/instance
I20260806 16:09:33.415723 8388 external_mini_cluster.cc:1428] Setting key a9d7445b2da85161952cf01441c5fbfd
I20260806 16:09:33.416638 27426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.49.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.60@KRBTEST.COM" created.
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:33.439458 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.419906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59648 (local address 127.8.49.62:38001)
0806 16:09:33.420074 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.420079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.421778 (+ 1699us) server_negotiation.cc:407] Connection header received
0806 16:09:33.423059 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.423064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.423135 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.423260 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.424810 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.425438 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.426872 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.427157 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.427285 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.432987 (+ 5702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.432994 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.432997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.433050 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.435749 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.436589 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.436597 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.436602 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.436749 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.437386 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.437394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.437399 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.437882 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.438035 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.438912 (+ 877us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.439155 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 16:09:33.440382 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.420743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:38001 (local address 127.0.0.1:59648)
0806 16:09:33.421533 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.421604 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:33.422727 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:33.423540 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:33.423549 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:33.423989 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:33.424628 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.424644 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.425605 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.425609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.426682 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.426694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.427362 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.427367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.427578 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.428395 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:33.428416 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:33.432610 (+ 4194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:33.435979 (+ 3369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.435988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.436009 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.436422 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.436889 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.436891 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.436893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.437079 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.438060 (+ 981us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:33.438066 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:33.438489 (+ 423us) client_negotiation.cc:769] Sending connection context
0806 16:09:33.438710 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 16:09:33.439152 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":207,"threads_started":1}
I20260806 16:09:33.442456 27426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab.
Entry for principal kudu/127.8.49.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab.
I20260806 16:09:33.456277 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.60:34785
--webserver_interface=127.8.49.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab
--principal=kudu/127.8.49.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:38001,127.8.49.61:34179,127.8.49.60:34785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:09:33.456796 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.443392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:34179 (local address 127.0.0.1:43146)
0806 16:09:33.444069 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.444089 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:33.444213 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:33.445097 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:33.445100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:33.445300 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:33.445437 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.445443 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.446784 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.446787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.447752 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.447765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.448854 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.448857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.448965 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.449715 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:33.449735 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:33.451587 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:33.454620 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.454624 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.454626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.455088 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.455551 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.455557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.455559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.455650 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.456189 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:33.456192 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:33.456280 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 16:09:33.456420 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 16:09:33.456614 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553}
I20260806 16:09:33.457027 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.443553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.8.49.61:34179)
0806 16:09:33.444261 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.444281 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.444325 (+ 44us) server_negotiation.cc:407] Connection header received
0806 16:09:33.444410 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.444421 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.444853 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.445071 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.445593 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.446639 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.447936 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.448692 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.448802 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.451810 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.451823 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.451844 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.451878 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.454398 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.455265 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.455274 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.455275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.455437 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.455752 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.455760 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.455762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.456076 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.456241 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.456681 (+ 440us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.456839 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":176,"threads_started":1}
I20260806 16:09:33.459487 27426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:09:33.460701 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.460260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49032)
0806 16:09:33.460465 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.460603 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260806 16:09:33.461056 27357 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
I20260806 16:09:33.462478 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.462118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49040)
0806 16:09:33.462275 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.462341 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 16:09:33.462908 27426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:34785: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
I20260806 16:09:33.512297 27426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } attempt: 1
I20260806 16:09:33.513428 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.513098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49054)
0806 16:09:33.513274 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.513346 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 16:09:33.514490 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.514237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49056)
0806 16:09:33.514376 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.514431 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 16:09:33.514773 27426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:34785: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
I20260806 16:09:33.518544 27323 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } attempt: 4
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
I20260806 16:09:33.530685 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.519172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:34179 (local address 127.0.0.1:43158)
0806 16:09:33.519365 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.519382 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:33.519487 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:33.519947 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:33.519952 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:33.520257 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:33.521016 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.521031 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.521976 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.521980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.522902 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.522915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.524174 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.524178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.524309 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.525023 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:33.525039 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:33.526944 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:33.528963 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.528967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.528970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.529217 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.529540 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.529543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.529545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.529636 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.530072 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:33.530076 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:33.530203 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 16:09:33.530317 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 16:09:33.530499 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 16:09:33.530685 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.519313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43158 (local address 127.8.49.61:34179)
0806 16:09:33.519641 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.519645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.519656 (+ 11us) server_negotiation.cc:407] Connection header received
0806 16:09:33.519695 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.519697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.519737 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.519804 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.521267 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.521840 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.523070 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.524002 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.524115 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.527167 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.527171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.527173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.527201 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.528795 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.529337 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.529340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.529342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.529428 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.529738 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.529742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.529744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.529952 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.530070 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.530376 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.530498 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247}
I20260806 16:09:33.532336 27323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:09:33.533332 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.533027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49070)
0806 16:09:33.533192 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.533259 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 16:09:33.534322 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.534076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49084)
0806 16:09:33.534202 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.534265 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 16:09:33.534600 27323 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:34785: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
I20260806 16:09:33.583031 27426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } attempt: 2
I20260806 16:09:33.584466 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.583948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49098)
0806 16:09:33.584206 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.584294 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 16:09:33.585598 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.585340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49106)
0806 16:09:33.585456 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.585517 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 16:09:33.585736 27426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:34785: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
I20260806 16:09:33.593797 27323 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } attempt: 1
I20260806 16:09:33.595227 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.594604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49112)
0806 16:09:33.595026 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.595120 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278}
I20260806 16:09:33.596292 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.595991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49124)
0806 16:09:33.596171 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.596233 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260806 16:09:33.596439 27323 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:34785: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
W20260806 16:09:33.615617 27435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:33.616101 27435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:33.616209 27435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:33.621713 27435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:09:33.621886 27435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:33.621932 27435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:33.621973 27435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:09:33.622005 27435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:09:33.629048 27435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:38001,127.8.49.61:34179,127.8.49.60:34785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab
--principal=kudu/127.8.49.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.60:34785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.49.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:33.630877 27435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:33.632329 27435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:33.644723 27442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:33.644723 27444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:33.644886 27441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:33.645239 27435 server_base.cc:1034] running on GCE node
I20260806 16:09:33.646323 27435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:33.648200 27435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:33.649451 27435 hybrid_clock.cc:648] HybridClock initialized: now 1786032573649409 us; error 65 us; skew 500 ppm
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:33.654044 27435 init.cc:377] Logged in from keytab as kudu/127.8.49.60@KRBTEST.COM (short username kudu)
I20260806 16:09:33.655390 27435 webserver.cc:460] Webserver started at http://127.8.49.60:38779/ using document root <none> and password file <none>
I20260806 16:09:33.656111 27435 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:33.656181 27435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:33.656399 27435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:09:33.659013 27435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/data/instance:
uuid: "a15118cea69b403f92d0982d16a6a618"
format_stamp: "Formatted at 2026-08-06 16:09:33 on dist-test-slave-7z32"
server_key: "c81b7111e36cb29138b9c0a17cbf82ec"
server_key_iv: "5d88a12ec31c0d165bee2315951f811c"
server_key_version: "encryptionkey@0"
I20260806 16:09:33.659636 27435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/wal/instance:
uuid: "a15118cea69b403f92d0982d16a6a618"
format_stamp: "Formatted at 2026-08-06 16:09:33 on dist-test-slave-7z32"
server_key: "c81b7111e36cb29138b9c0a17cbf82ec"
server_key_iv: "5d88a12ec31c0d165bee2315951f811c"
server_key_version: "encryptionkey@0"
I20260806 16:09:33.663622 27323 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } attempt: 2
I20260806 16:09:33.664625 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.664325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49132)
0806 16:09:33.664475 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.664549 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 16:09:33.664891 27435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:09:33.665729 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.665500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49136)
0806 16:09:33.665619 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.665672 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 16:09:33.665884 27323 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:34785: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
I20260806 16:09:33.668896 27453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:33.671083 27435 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:09:33.671291 27435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/wal
uuid: "a15118cea69b403f92d0982d16a6a618"
format_stamp: "Formatted at 2026-08-06 16:09:33 on dist-test-slave-7z32"
server_key: "c81b7111e36cb29138b9c0a17cbf82ec"
server_key_iv: "5d88a12ec31c0d165bee2315951f811c"
server_key_version: "encryptionkey@0"
I20260806 16:09:33.671461 27435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:33.676909 27426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } attempt: 3
I20260806 16:09:33.677917 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.677576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49146)
0806 16:09:33.677742 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.677823 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 16:09:33.679210 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.678861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49148)
0806 16:09:33.679065 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.679130 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260806 16:09:33.679384 27426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:34785: Network error: Client connection negotiation failed: client connection to 127.8.49.60:34785: connect: Connection refused (error 111)
I20260806 16:09:33.688427 27435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:33.693650 27435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:33.693981 27435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:33.704535 27435 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.60:34785
I20260806 16:09:33.704599 27531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.60:34785 every 8 connection(s)
I20260806 16:09:33.706208 27435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
I20260806 16:09:33.709764 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 27435
I20260806 16:09:33.709966 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786032364843570-8388-0/minicluster-data/master-2/wal/instance
I20260806 16:09:33.710160 27532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:33.710431 8388 external_mini_cluster.cc:1428] Setting key e2315b3bc94698bb1293ea8b5695a8c6
I20260806 16:09:33.715351 27532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:33.731045 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.717737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.8.49.62:38001)
0806 16:09:33.717943 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.717949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.718319 (+ 370us) server_negotiation.cc:407] Connection header received
0806 16:09:33.718525 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.718529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.718591 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.718708 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.719884 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.720671 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.721994 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.722333 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.722443 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.725816 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.725822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.725828 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.725873 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.727946 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.728691 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.728695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.728697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.728813 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.729420 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.729423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.729425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.729740 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.729878 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.730593 (+ 715us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.730763 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 16:09:33.732288 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.717576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:38001 (local address 127.0.0.1:59660)
0806 16:09:33.718159 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.718214 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:33.718399 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:33.718902 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:33.718954 (+ 52us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:33.719494 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:33.719686 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.719695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.720870 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.720874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.721820 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.721832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.722571 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.722577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.722729 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.723507 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:33.723527 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:33.725569 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:33.728104 (+ 2535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.728120 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.728140 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.728535 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.729043 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.729049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.729051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.729273 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.729905 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:33.729914 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:33.730137 (+ 223us) client_negotiation.cc:769] Sending connection context
0806 16:09:33.730439 (+ 302us) client_negotiation.cc:240] Negotiation successful
0806 16:09:33.731157 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":9,"thread_start_us":231,"threads_started":1}
I20260806 16:09:33.732300 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.713250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49150 (local address 127.8.49.60:34785)
0806 16:09:33.713948 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.713970 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.714023 (+ 53us) server_negotiation.cc:407] Connection header received
0806 16:09:33.715025 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.715053 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.715554 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.716040 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.717140 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.718620 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.720066 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.720967 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.721118 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.725055 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.725063 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.725079 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.725106 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.727815 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.728715 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.728723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.728724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.728891 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.729498 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.729501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.729503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.730111 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.730308 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.730688 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.731365 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":340,"server-negotiator.queue_time_us":480,"thread_start_us":239,"threads_started":1}
I20260806 16:09:33.734005 27532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:33.746433 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.734938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.8.49.61:34179)
0806 16:09:33.735114 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.735119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.735136 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:09:33.735258 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.735263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.735311 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.735403 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.735966 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.736544 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.737881 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.738240 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.738383 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.741954 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.741965 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.741968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.742006 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.744110 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.744761 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.744765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.744767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.744872 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.745208 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.745212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.745213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.745468 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.745586 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.746046 (+ 460us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.746212 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:09:33.746433 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.734649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:34179 (local address 127.0.0.1:43160)
0806 16:09:33.735007 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.735029 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:33.735143 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:33.735498 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:33.735502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:33.735679 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:33.735820 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.735827 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.736690 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.736693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.737663 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.737676 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.738520 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.738528 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.738662 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.739443 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:33.739461 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:33.741661 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:33.744265 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.744270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.744272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.744647 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.744995 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.744999 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.745001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.745102 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.745573 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:33.745577 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:33.745700 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 16:09:33.745980 (+ 280us) client_negotiation.cc:240] Negotiation successful
0806 16:09:33.746214 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235}
I20260806 16:09:33.747460 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.735849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.8.49.62:38001)
0806 16:09:33.736743 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.736747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.736760 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:09:33.736802 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.736805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.736854 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.736922 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.737881 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.738628 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.739953 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.740260 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.740396 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.743640 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.743645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.743647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.743683 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.745236 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.745737 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.745741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.745743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.745835 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.746169 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.746172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.746174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.746424 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.747022 (+ 598us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.747153 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.747289 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791}
I20260806 16:09:33.748082 27532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
I20260806 16:09:33.759311 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.748907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49166)
0806 16:09:33.749070 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.749088 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:33.749222 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:33.749468 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:33.749768 (+ 300us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:33.749985 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:33.750093 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.750100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.750869 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.750875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.751412 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.751419 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.751903 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.751907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.751975 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.752525 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:33.752545 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:33.754778 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:33.757233 (+ 2455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.757238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.757241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.757536 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.757892 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.757894 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.757896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.757985 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.758605 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:33.758608 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:33.758750 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 16:09:33.758944 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 16:09:33.759128 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 16:09:33.759486 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.748967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49166 (local address 127.8.49.60:34785)
0806 16:09:33.749083 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.749088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.749162 (+ 74us) server_negotiation.cc:407] Connection header received
0806 16:09:33.749322 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.749325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.749364 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.749456 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.750191 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.750727 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.751539 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.751790 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.751903 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.755014 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.755018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.755021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.755045 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.757031 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.757649 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.757653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.757655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.757767 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.758105 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.758108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.758110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.758473 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.758629 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.759037 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.759219 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:09:33.760651 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.749672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43166 (local address 127.8.49.61:34179)
0806 16:09:33.749810 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.749817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.749834 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:09:33.749881 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.749884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.749938 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.750044 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.750888 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.751440 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.752266 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.752514 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.752632 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.756302 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.756309 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.756314 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.756363 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.758390 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.759053 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.759057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.759058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.759158 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.759553 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.759560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.759565 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.759850 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.760214 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.760296 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.760440 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:09:33.763808 27532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618: Bootstrap starting.
I20260806 16:09:33.766067 27323 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } attempt: 3
I20260806 16:09:33.767897 27532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:33.768078 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:09:33.769140 27532 log.cc:824] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
I20260806 16:09:33.773541 27532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618: No bootstrap required, opened a new log
I20260806 16:09:33.777619 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.766628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49182)
0806 16:09:33.766757 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.766774 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:33.766909 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:33.767394 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:33.767398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:33.767689 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:33.767884 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.767894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.768739 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.768743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.769304 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.769313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.769846 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.769850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.769944 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.770544 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:33.770557 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:33.773059 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:33.775284 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.775288 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.775290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.775611 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.775941 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.775944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.775946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.776077 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.776713 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:33.776716 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:33.777018 (+ 302us) client_negotiation.cc:769] Sending connection context
0806 16:09:33.777139 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 16:09:33.777359 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":173}
I20260806 16:09:33.778306 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.766959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49182 (local address 127.8.49.60:34785)
0806 16:09:33.767117 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.767121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.767136 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:09:33.767178 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.767181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.767229 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.767318 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.768049 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.768606 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.769435 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.769719 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.769840 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.773254 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.773261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.773268 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.773303 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.775150 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.775726 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.775729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.775731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.775818 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.776187 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.776190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.776192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.776573 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.776666 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.777961 (+ 1295us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.778128 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:09:33.780121 27532 raft_consensus.cc:348] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } }
I20260806 16:09:33.780643 27532 raft_consensus.cc:374] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:33.780766 27532 raft_consensus.cc:729] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a15118cea69b403f92d0982d16a6a618, State: Initialized, Role: FOLLOWER
I20260806 16:09:33.781545 27532 consensus_queue.cc:260] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } }
I20260806 16:09:33.782428 27323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a: Bootstrap starting.
I20260806 16:09:33.783535 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.772191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.8.49.61:34179)
0806 16:09:33.772357 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.772367 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.772398 (+ 31us) server_negotiation.cc:407] Connection header received
0806 16:09:33.772459 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.772463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.772520 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.773684 (+ 1164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.773818 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.774675 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.776079 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.776417 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.778524 (+ 2107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.778658 (+ 134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.778661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.778664 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.778683 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.781068 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.781649 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.781652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.781654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.781743 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.782065 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.782068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.782070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.782296 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.783161 (+ 865us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.783240 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.783367 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 16:09:33.784106 27532 sys_catalog.cc:564] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:09:33.784894 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.770978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.8.49.62:38001)
0806 16:09:33.771109 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.771115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.772371 (+ 1256us) server_negotiation.cc:407] Connection header received
0806 16:09:33.772617 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.772622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.772688 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.772747 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.774100 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.774903 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.776302 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.776638 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.776767 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.780488 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.780492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.780497 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.780533 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.782666 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.783330 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.783334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.783336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.783434 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.783763 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.783766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.783768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.784032 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.784160 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.784552 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.784713 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":49}
I20260806 16:09:33.786950 27323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:33.787976 27551 sys_catalog.cc:455] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } } }
I20260806 16:09:33.788200 27551 sys_catalog.cc:458] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 16:09:33.791281 27564 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:09:33.791471 27564 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:09:33.792500 27323 log.cc:824] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:33.788748 27435 master_runner.cc:186] Error getting master registration for 127.8.49.62:38001: OK, instance_id { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" instance_seqno: 1786032573116591 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:09:33.793026 27435 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:09:33.796548 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.772492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.8.49.60:34785)
0806 16:09:33.777042 (+ 4550us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.777058 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.777371 (+ 313us) server_negotiation.cc:407] Connection header received
0806 16:09:33.777602 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.777608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.777675 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.778151 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.787828 (+ 9677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.788724 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.789897 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.790341 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.790464 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.792412 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.792416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.792419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.792448 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.794384 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.794921 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.794925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.794927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.795045 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.795361 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.795364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.795366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.795583 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.795774 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.796057 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.796214 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4451,"thread_start_us":129,"threads_started":1}
I20260806 16:09:33.796608 27323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a: No bootstrap required, opened a new log
I20260806 16:09:33.802188 27323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } }
I20260806 16:09:33.802614 27323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:33.802680 27323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 310cc0dd23ce43a4b739f3938bbd831a, State: Initialized, Role: FOLLOWER
I20260806 16:09:33.803448 27323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } }
I20260806 16:09:33.804877 27567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } } }
I20260806 16:09:33.805143 27567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:09:33.805754 27323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [sys.catalog]: configured and running, proceeding with master startup.
W20260806 16:09:33.814862 27583 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:09:33.815143 27583 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:09:33.812384 27217 master_runner.cc:186] Error getting master registration for 127.8.49.61:34179: OK, instance_id { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" instance_seqno: 1786032573402229 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:09:33.817234 27217 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:09:33.822594 27426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } attempt: 4
Aug 06 16:09:33 dist-test-slave-7z32 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032573, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
I20260806 16:09:33.835412 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.823327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:34785 (local address 127.0.0.1:49198)
0806 16:09:33.823857 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:33.823876 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:33.823987 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:33.824616 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:33.824619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:33.824879 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:33.825020 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.825028 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.826258 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.826263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.827266 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:33.827283 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.828011 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.828015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:33.828119 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.828856 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:33.828873 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:33.830816 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:33.833482 (+ 2666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.833486 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.833489 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.833767 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.834099 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:33.834102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:33.834104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:33.834191 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:33.834739 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:33.834742 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:33.834906 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 16:09:33.835032 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 16:09:33.835222 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427}
I20260806 16:09:33.836129 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:33.823937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.8.49.60:34785)
0806 16:09:33.824198 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:33.824203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:33.824220 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:09:33.824277 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:33.824283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:33.824341 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:33.824424 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:33.825188 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.826073 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.827441 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:33.827842 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:33.827976 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:33.831081 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:33.831087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:33.831090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:33.831133 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:33.833323 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.833886 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.833890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.833892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.833980 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:33.834289 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:33.834293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:33.834294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:33.834535 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:33.834659 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:33.835884 (+ 1225us) server_negotiation.cc:299] Negotiation successful
0806 16:09:33.836007 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260806 16:09:33.839784 27426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de: Bootstrap starting.
I20260806 16:09:33.842314 27426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:33.843261 27426 log.cc:824] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:33.845827 27426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de: No bootstrap required, opened a new log
I20260806 16:09:33.849076 27426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } }
I20260806 16:09:33.849308 27426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:33.849404 27426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84174475343a427799b0c1a06c0bf2de, State: Initialized, Role: FOLLOWER
I20260806 16:09:33.849928 27426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } }
I20260806 16:09:33.850462 27586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } } }
I20260806 16:09:33.850646 27586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:09:33.851615 27426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [sys.catalog]: configured and running, proceeding with master startup.
W20260806 16:09:33.861997 27247 tablet.cc:2369] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:09:33.863022 27330 master_runner.cc:263] No leader master found from master 84174475343a427799b0c1a06c0bf2de
I20260806 16:09:33.863106 27330 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 16:09:33.864041 27598 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:09:33.864212 27598 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:09:33.884915 27586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:09:33.885165 27586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } }
I20260806 16:09:33.886389 27586 leader_election.cc:288] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 310cc0dd23ce43a4b739f3938bbd831a (127.8.49.62:38001), a15118cea69b403f92d0982d16a6a618 (127.8.49.60:34785)
I20260806 16:09:33.887123 27493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84174475343a427799b0c1a06c0bf2de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a15118cea69b403f92d0982d16a6a618" is_pre_election: true
I20260806 16:09:33.887475 27493 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84174475343a427799b0c1a06c0bf2de in term 0.
I20260806 16:09:33.888105 27279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84174475343a427799b0c1a06c0bf2de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "310cc0dd23ce43a4b739f3938bbd831a" is_pre_election: true
I20260806 16:09:33.888417 27279 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84174475343a427799b0c1a06c0bf2de in term 0.
I20260806 16:09:33.889145 27357 leader_election.cc:302] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84174475343a427799b0c1a06c0bf2de, a15118cea69b403f92d0982d16a6a618; no voters:
I20260806 16:09:33.889452 27586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:09:33.889535 27586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:09:33.889595 27586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:33.890867 27586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } }
I20260806 16:09:33.891314 27586 leader_election.cc:288] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [CANDIDATE]: Term 1 election: Requested vote from peers 310cc0dd23ce43a4b739f3938bbd831a (127.8.49.62:38001), a15118cea69b403f92d0982d16a6a618 (127.8.49.60:34785)
I20260806 16:09:33.891983 27279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84174475343a427799b0c1a06c0bf2de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "310cc0dd23ce43a4b739f3938bbd831a"
I20260806 16:09:33.892156 27279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:33.893632 27279 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84174475343a427799b0c1a06c0bf2de in term 1.
I20260806 16:09:33.894130 27493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84174475343a427799b0c1a06c0bf2de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a15118cea69b403f92d0982d16a6a618"
I20260806 16:09:33.894313 27493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [term 0 FOLLOWER]: Advancing to term 1
W20260806 16:09:33.897331 27362 tablet.cc:2369] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:09:33.897671 27493 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84174475343a427799b0c1a06c0bf2de in term 1.
I20260806 16:09:33.899408 27357 leader_election.cc:302] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84174475343a427799b0c1a06c0bf2de, a15118cea69b403f92d0982d16a6a618; no voters:
I20260806 16:09:33.899899 27586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:09:33.900112 27586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [term 1 LEADER]: Becoming Leader. State: Replica: 84174475343a427799b0c1a06c0bf2de, State: Running, Role: LEADER
I20260806 16:09:33.900491 27586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } }
I20260806 16:09:33.902410 27586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84174475343a427799b0c1a06c0bf2de. Latest consensus state: current_term: 1 leader_uuid: "84174475343a427799b0c1a06c0bf2de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } } }
I20260806 16:09:33.902520 27586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:33.907263 27605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:09:33.908792 27605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:09:33.921875 27279 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [term 1 FOLLOWER]: Refusing update from remote peer 84174475343a427799b0c1a06c0bf2de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:09:33.923080 27493 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [term 1 FOLLOWER]: Refusing update from remote peer 84174475343a427799b0c1a06c0bf2de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:09:33.924149 27586 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [LEADER]: Connected to new peer: Peer: permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:09:33.924616 27602 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [LEADER]: Connected to new peer: Peer: permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:09:33.930223 27551 sys_catalog.cc:455] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84174475343a427799b0c1a06c0bf2de. Latest consensus state: current_term: 1 leader_uuid: "84174475343a427799b0c1a06c0bf2de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } } }
I20260806 16:09:33.930413 27551 sys_catalog.cc:458] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:09:33.939692 27567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84174475343a427799b0c1a06c0bf2de. Latest consensus state: current_term: 1 leader_uuid: "84174475343a427799b0c1a06c0bf2de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } } }
I20260806 16:09:33.939848 27567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:09:33.945375 27602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "84174475343a427799b0c1a06c0bf2de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } } }
I20260806 16:09:33.945573 27602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [sys.catalog]: This master's current role is: LEADER
W20260806 16:09:33.946596 27463 tablet.cc:2369] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:09:33.947144 27567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "84174475343a427799b0c1a06c0bf2de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } } }
I20260806 16:09:33.947278 27567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:09:33.948045 27610 mvcc.cc:205] Tried to move back new op lower bound from 7315589422773719040 to 7315589422699720704. Current Snapshot: MvccSnapshot[applied={T|T < 7315589422773719040}]
I20260806 16:09:33.949214 27586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "84174475343a427799b0c1a06c0bf2de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } } }
I20260806 16:09:33.949352 27586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:33.950902 27551 sys_catalog.cc:455] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "84174475343a427799b0c1a06c0bf2de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "310cc0dd23ce43a4b739f3938bbd831a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 38001 } } peers { permanent_uuid: "84174475343a427799b0c1a06c0bf2de" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 34179 } } peers { permanent_uuid: "a15118cea69b403f92d0982d16a6a618" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 34785 } } }
I20260806 16:09:33.951040 27551 sys_catalog.cc:458] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:09:33.954418 27605 catalog_manager.cc:1347] Generated new cluster ID: 90657df04bfd44b8873cf01fc10829ed
I20260806 16:09:33.954494 27605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:09:33.975006 27605 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:09:33.980329 27605 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:09:34.006649 27605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84174475343a427799b0c1a06c0bf2de: Generated new TSK 0
I20260806 16:09:34.008011 27605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:09:34.793524 27564 catalog_manager.cc:1259] Loaded cluster ID: 90657df04bfd44b8873cf01fc10829ed
I20260806 16:09:34.793660 27564 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618: loading cluster ID for follower catalog manager: success
I20260806 16:09:34.796068 27564 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618: acquiring CA information for follower catalog manager: success
I20260806 16:09:34.797740 27564 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a15118cea69b403f92d0982d16a6a618: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:09:34.817559 27583 catalog_manager.cc:1259] Loaded cluster ID: 90657df04bfd44b8873cf01fc10829ed
I20260806 16:09:34.817723 27583 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a: loading cluster ID for follower catalog manager: success
I20260806 16:09:34.820209 27583 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a: acquiring CA information for follower catalog manager: success
I20260806 16:09:34.822016 27583 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 310cc0dd23ce43a4b739f3938bbd831a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:09:35.105090 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 27217
I20260806 16:09:35.125505 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 27330
I20260806 16:09:35.145251 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 27435
2026-08-06T16:09:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3820 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:09:35 dist-test-slave-7z32 krb5kdc[27632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:09:35 dist-test-slave-7z32 krb5kdc[27632](info): set up 2 sockets
Aug 06 16:09:35 dist-test-slave-7z32 krb5kdc[27632](info): commencing operation
krb5kdc: starting...
W20260806 16:09:36.476110 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:09:36 dist-test-slave-7z32 krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:09:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:09:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:09:36.643359 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42647
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:42647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:36.778221 27650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:36.778592 27650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:36.778638 27650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:36.783052 27650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:09:36.783180 27650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:36.783250 27650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:36.783282 27650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:09:36.783321 27650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:09:36.789161 27650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:42647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:36.790524 27650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:36.791733 27650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:36.803262 27658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:36.803363 27656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:36.803363 27655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:36.804121 27650 server_base.cc:1034] running on GCE node
I20260806 16:09:36.805006 27650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:36.806557 27650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:36.810055 27650 hybrid_clock.cc:648] HybridClock initialized: now 1786032576809975 us; error 87 us; skew 500 ppm
Aug 06 16:09:36 dist-test-slave-7z32 krb5kdc[27632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032576, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:36.814347 27650 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:09:36.815698 27650 webserver.cc:460] Webserver started at http://127.8.49.62:36547/ using document root <none> and password file <none>
I20260806 16:09:36.816363 27650 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:36.816464 27650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:36.816730 27650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:09:36.819203 27650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "c1186b3fcd3e4f5ca0960819b19c4f91"
format_stamp: "Formatted at 2026-08-06 16:09:36 on dist-test-slave-7z32"
server_key: "2c43e57cbd4128422ad63c9b90a61fa1"
server_key_iv: "7bf56916f5156ac0faed6e59c9c4b79d"
server_key_version: "encryptionkey@0"
I20260806 16:09:36.819880 27650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "c1186b3fcd3e4f5ca0960819b19c4f91"
format_stamp: "Formatted at 2026-08-06 16:09:36 on dist-test-slave-7z32"
server_key: "2c43e57cbd4128422ad63c9b90a61fa1"
server_key_iv: "7bf56916f5156ac0faed6e59c9c4b79d"
server_key_version: "encryptionkey@0"
I20260806 16:09:36.825141 27650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:09:36.829250 27670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:36.831418 27650 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:09:36.831671 27650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "c1186b3fcd3e4f5ca0960819b19c4f91"
format_stamp: "Formatted at 2026-08-06 16:09:36 on dist-test-slave-7z32"
server_key: "2c43e57cbd4128422ad63c9b90a61fa1"
server_key_iv: "7bf56916f5156ac0faed6e59c9c4b79d"
server_key_version: "encryptionkey@0"
I20260806 16:09:36.831864 27650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:36.856413 27650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:36.861641 27650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:36.861893 27650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:36.872323 27650 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:42647
I20260806 16:09:36.872449 27737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:42647 every 8 connection(s)
I20260806 16:09:36.873872 27650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:09:36.875559 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 27650
I20260806 16:09:36.875754 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:09:36.876133 8388 external_mini_cluster.cc:1428] Setting key 0669cf56976b026800fc16b1ba8c358b
I20260806 16:09:36.878170 27738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:36.886086 27738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91: Bootstrap starting.
Aug 06 16:09:36 dist-test-slave-7z32 krb5kdc[27632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:36.888875 27738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:36.889868 27738 log.cc:824] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:36.892966 27738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91: No bootstrap required, opened a new log
I20260806 16:09:36.895435 27742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:36.878042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51732 (local address 127.8.49.62:42647)
0806 16:09:36.878745 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:36.878767 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:36.878847 (+ 80us) server_negotiation.cc:407] Connection header received
0806 16:09:36.879859 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:36.879885 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:36.880369 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:36.880807 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:36.881896 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:36.883514 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:36.884657 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:36.885417 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:36.885610 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:36.889234 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:36.889241 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:36.889260 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:36.889304 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:36.891562 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:36.892127 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:36.892133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:36.892139 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:36.892293 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:36.892726 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:36.892730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:36.892731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:36.893319 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:36.893529 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:36.893968 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 16:09:36.894322 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":264,"threads_started":1}
I20260806 16:09:36.897837 27738 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1186b3fcd3e4f5ca0960819b19c4f91" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42647 } }
I20260806 16:09:36.898180 27738 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:36.898236 27738 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1186b3fcd3e4f5ca0960819b19c4f91, State: Initialized, Role: FOLLOWER
I20260806 16:09:36.898751 27738 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1186b3fcd3e4f5ca0960819b19c4f91" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42647 } }
I20260806 16:09:36.898957 27738 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:09:36.899055 27738 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:09:36.899168 27738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:36.900328 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:09:36.900496 27738 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1186b3fcd3e4f5ca0960819b19c4f91" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42647 } }
I20260806 16:09:36.900889 27738 leader_election.cc:302] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1186b3fcd3e4f5ca0960819b19c4f91; no voters:
I20260806 16:09:36.901445 27738 leader_election.cc:288] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:09:36.901710 27745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:09:36.902096 27745 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [term 1 LEADER]: Becoming Leader. State: Replica: c1186b3fcd3e4f5ca0960819b19c4f91, State: Running, Role: LEADER
I20260806 16:09:36.902621 27745 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1186b3fcd3e4f5ca0960819b19c4f91" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42647 } }
I20260806 16:09:36.903280 27738 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:09:36.904389 27745 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1186b3fcd3e4f5ca0960819b19c4f91. Latest consensus state: current_term: 1 leader_uuid: "c1186b3fcd3e4f5ca0960819b19c4f91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1186b3fcd3e4f5ca0960819b19c4f91" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42647 } } }
I20260806 16:09:36.904692 27745 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:36.905052 27750 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1186b3fcd3e4f5ca0960819b19c4f91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1186b3fcd3e4f5ca0960819b19c4f91" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42647 } } }
I20260806 16:09:36.905181 27750 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91 [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:36.905342 27759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:09:36.909335 27759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:09:36.917680 27759 catalog_manager.cc:1347] Generated new cluster ID: e15c22202cd4429f9fbed870937e7a8c
I20260806 16:09:36.917826 27759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:09:36.942193 27759 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:09:36.943516 27759 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:09:36.950297 27759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1186b3fcd3e4f5ca0960819b19c4f91: Generated new TSK 0
I20260806 16:09:36.951284 27759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:09:36.959230 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 27650
2026-08-06T16:09:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1812 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:09:37 dist-test-slave-7z32 krb5kdc[27778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:09:37 dist-test-slave-7z32 krb5kdc[27778](info): set up 2 sockets
Aug 06 16:09:37 dist-test-slave-7z32 krb5kdc[27778](info): commencing operation
krb5kdc: starting...
W20260806 16:09:38.302757 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:09:38 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:09:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:09:38.475977 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:37299
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:37299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:38.635715 27799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:38.636130 27799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:38.636215 27799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:38.640662 27799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:09:38.640802 27799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:38.640841 27799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:38.640882 27799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:09:38.640920 27799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:09:38.649549 27799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:37299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:37299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:38.650990 27799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:38.652185 27799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:09:38.663220 27799 server_base.cc:1034] running on GCE node
W20260806 16:09:38.663092 27805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:38.663080 27806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:38.663080 27809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:38.664392 27799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:38.666021 27799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:38.669499 27799 hybrid_clock.cc:648] HybridClock initialized: now 1786032578669420 us; error 89 us; skew 500 ppm
Aug 06 16:09:38 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032578, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:38.673594 27799 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:09:38.675034 27799 webserver.cc:460] Webserver started at http://127.8.49.62:40631/ using document root <none> and password file <none>
I20260806 16:09:38.676208 27799 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:38.676301 27799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:38.676589 27799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:09:38.679567 27799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "a103b579a9b34f27a98f92ab8f93c23e"
format_stamp: "Formatted at 2026-08-06 16:09:38 on dist-test-slave-7z32"
server_key: "7c9c83892bf8e542e7b9249d73a50371"
server_key_iv: "c08a78060b2e89c90a2a5d1de6a229e8"
server_key_version: "encryptionkey@0"
I20260806 16:09:38.680462 27799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "a103b579a9b34f27a98f92ab8f93c23e"
format_stamp: "Formatted at 2026-08-06 16:09:38 on dist-test-slave-7z32"
server_key: "7c9c83892bf8e542e7b9249d73a50371"
server_key_iv: "c08a78060b2e89c90a2a5d1de6a229e8"
server_key_version: "encryptionkey@0"
I20260806 16:09:38.686276 27799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:09:38.690126 27818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:38.692076 27799 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:09:38.692271 27799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "a103b579a9b34f27a98f92ab8f93c23e"
format_stamp: "Formatted at 2026-08-06 16:09:38 on dist-test-slave-7z32"
server_key: "7c9c83892bf8e542e7b9249d73a50371"
server_key_iv: "c08a78060b2e89c90a2a5d1de6a229e8"
server_key_version: "encryptionkey@0"
I20260806 16:09:38.692476 27799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:38.709779 27799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:38.715353 27799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:38.715559 27799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:38.727044 27799 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:37299
I20260806 16:09:38.727514 27885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:37299 every 8 connection(s)
I20260806 16:09:38.728379 27799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:09:38.731014 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 27799
I20260806 16:09:38.731137 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:09:38.731469 8388 external_mini_cluster.cc:1428] Setting key 56b6a9a301d2cf68cd930eb7598f295b
I20260806 16:09:38.733403 27888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:38.744024 27888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Bootstrap starting.
Aug 06 16:09:38 dist-test-slave-7z32 krb5kdc[27778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:38.748409 27888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:38.749953 27888 log.cc:824] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:38.753119 27888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: No bootstrap required, opened a new log
I20260806 16:09:38.753504 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:38.733669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.8.49.62:37299)
0806 16:09:38.734343 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:38.734370 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:38.734435 (+ 65us) server_negotiation.cc:407] Connection header received
0806 16:09:38.735628 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:38.735660 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:38.736249 (+ 589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:38.736809 (+ 560us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:38.737920 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:38.739835 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:38.741035 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:38.741878 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:38.742204 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:38.745348 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:38.745357 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:38.745381 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:38.745438 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:38.748334 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:38.749437 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:38.749445 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:38.749454 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:38.749701 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:38.750374 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:38.750381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:38.750383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:38.751192 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:38.751471 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:38.752018 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 16:09:38.752465 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":190,"threads_started":1}
I20260806 16:09:38.758600 27888 raft_consensus.cc:348] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:38.759104 27888 raft_consensus.cc:374] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:38.759179 27888 raft_consensus.cc:729] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a103b579a9b34f27a98f92ab8f93c23e, State: Initialized, Role: FOLLOWER
I20260806 16:09:38.759893 27888 consensus_queue.cc:260] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:38.760106 27888 raft_consensus.cc:388] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:09:38.760192 27888 raft_consensus.cc:482] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:09:38.760352 27888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:38.761933 27888 raft_consensus.cc:504] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:38.762542 27888 leader_election.cc:302] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a103b579a9b34f27a98f92ab8f93c23e; no voters:
I20260806 16:09:38.763232 27888 leader_election.cc:288] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:09:38.763567 27894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:09:38.764204 27894 raft_consensus.cc:686] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 1 LEADER]: Becoming Leader. State: Replica: a103b579a9b34f27a98f92ab8f93c23e, State: Running, Role: LEADER
I20260806 16:09:38.764690 27894 consensus_queue.cc:237] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:38.765229 27888 sys_catalog.cc:564] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:09:38.766778 27895 sys_catalog.cc:455] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a103b579a9b34f27a98f92ab8f93c23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } } }
I20260806 16:09:38.766991 27895 sys_catalog.cc:458] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:38.767462 27898 sys_catalog.cc:455] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a103b579a9b34f27a98f92ab8f93c23e. Latest consensus state: current_term: 1 leader_uuid: "a103b579a9b34f27a98f92ab8f93c23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } } }
I20260806 16:09:38.767577 27898 sys_catalog.cc:458] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:38.768028 27912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:09:38.773146 27912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:09:38.782106 27912 catalog_manager.cc:1347] Generated new cluster ID: 0372b8b360e544ca94522b722cd0c28d
I20260806 16:09:38.782248 27912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:09:38.808153 27912 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:09:38.810178 27912 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:09:38.825850 27912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Generated new TSK 0
I20260806 16:09:38.827066 27912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:09:38.885360 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37299
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:39.040736 27925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:39.041122 27925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:39.041172 27925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:39.045764 27925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:39.045902 27925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:39.045977 27925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:09:39.052448 27925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:39.054344 27925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:39.056205 27925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:39.070995 27930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:39.071023 27933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:39.071023 27935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:39.071909 27925 server_base.cc:1034] running on GCE node
I20260806 16:09:39.072944 27925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:39.073791 27925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:39.077122 27925 hybrid_clock.cc:648] HybridClock initialized: now 1786032579077071 us; error 68 us; skew 500 ppm
Aug 06 16:09:39 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:39.081094 27925 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:09:39.082446 27925 webserver.cc:460] Webserver started at http://127.8.49.1:45469/ using document root <none> and password file <none>
I20260806 16:09:39.083415 27925 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:39.083523 27925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:39.083797 27925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:09:39.086483 27925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "9e7a1e75b34142e18f120d73e01de516"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "d5c1a125ee1a6a3ea67fbeaeab090f73"
server_key_iv: "adea759ac7f4400327a0580ca2eb4d82"
server_key_version: "encryptionkey@0"
I20260806 16:09:39.087306 27925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "9e7a1e75b34142e18f120d73e01de516"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "d5c1a125ee1a6a3ea67fbeaeab090f73"
server_key_iv: "adea759ac7f4400327a0580ca2eb4d82"
server_key_version: "encryptionkey@0"
I20260806 16:09:39.092695 27925 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:09:39.096505 27947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:39.098692 27925 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:09:39.099022 27925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "9e7a1e75b34142e18f120d73e01de516"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "d5c1a125ee1a6a3ea67fbeaeab090f73"
server_key_iv: "adea759ac7f4400327a0580ca2eb4d82"
server_key_version: "encryptionkey@0"
I20260806 16:09:39.099205 27925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:39.121735 27925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:39.126309 27925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:39.126605 27925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:39.127424 27925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:39.128450 27925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:39.128526 27925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:39.128635 27925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:39.128686 27925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:39.141952 27925 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:40719
I20260806 16:09:39.142118 28104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:40719 every 8 connection(s)
I20260806 16:09:39.143260 27925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:09:39.148949 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 27925
I20260806 16:09:39.149150 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:09:39.149573 8388 external_mini_cluster.cc:1428] Setting key ffeb8b0fc43040148c55948481232559
Aug 06 16:09:39 dist-test-slave-7z32 krb5kdc[27778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
I20260806 16:09:39.167148 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.146548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:48139 (local address 127.8.49.62:37299)
0806 16:09:39.146931 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:39.146936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:39.149242 (+ 2306us) server_negotiation.cc:407] Connection header received
0806 16:09:39.150658 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:39.150662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:39.150746 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:39.150924 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:39.152936 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.153546 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.155715 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.156012 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.156163 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.160792 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:39.160797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:39.160799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:39.160835 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:39.163266 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:39.164487 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:39.164491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:39.164492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:39.164645 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:39.165236 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:39.165239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:39.165240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:39.165602 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:39.165707 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:39.166735 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0806 16:09:39.166990 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:09:39.168865 28107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.147125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:48139)
0806 16:09:39.148798 (+ 1673us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:39.148924 (+ 126us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:39.150328 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:39.151190 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:39.151200 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:39.151763 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:39.152691 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.152721 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.153764 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.153771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.155509 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.155525 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.156248 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.156253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.156473 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.157417 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:39.157452 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:39.160441 (+ 2989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:39.163543 (+ 3102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:39.163561 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:39.163604 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:39.164247 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:39.164780 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:39.164786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:39.164791 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:39.165083 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:39.165754 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:39.165763 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:39.166282 (+ 519us) client_negotiation.cc:769] Sending connection context
0806 16:09:39.166640 (+ 358us) client_negotiation.cc:240] Negotiation successful
0806 16:09:39.167205 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1212,"thread_start_us":695,"threads_started":1}
I20260806 16:09:39.171010 28105 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37299
I20260806 16:09:39.171398 28105 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:39.172137 28105 heartbeater.cc:511] Master 127.8.49.62:37299 requested a full tablet report, sending...
I20260806 16:09:39.174551 27838 ts_manager.cc:194] Registered new tserver with Master: 9e7a1e75b34142e18f120d73e01de516 (127.8.49.1:40719)
I20260806 16:09:39.176977 27838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:48139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:09:39.196237 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37299
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:39.345132 28110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:39.345482 28110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:39.345532 28110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:39.350487 28110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:39.350677 28110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:39.350757 28110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:09:39.357757 28110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:39.359251 28110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:39.360550 28110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:39.373167 28118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:39.373268 28117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:39.373268 28120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:39.373821 28110 server_base.cc:1034] running on GCE node
I20260806 16:09:39.374316 28110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:39.375133 28110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:39.376466 28110 hybrid_clock.cc:648] HybridClock initialized: now 1786032579376440 us; error 50 us; skew 500 ppm
Aug 06 16:09:39 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:39.380519 28110 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:09:39.381748 28110 webserver.cc:460] Webserver started at http://127.8.49.2:45921/ using document root <none> and password file <none>
I20260806 16:09:39.382489 28110 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:39.382599 28110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:39.383004 28110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:09:39.386349 28110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "4f247e66715a4fc6a7781a5e381ca821"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "3449c10e7c98720afd93c6a4ee768e0b"
server_key_iv: "886f34b0ccf72398b7dc2162782333f5"
server_key_version: "encryptionkey@0"
I20260806 16:09:39.387637 28110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "4f247e66715a4fc6a7781a5e381ca821"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "3449c10e7c98720afd93c6a4ee768e0b"
server_key_iv: "886f34b0ccf72398b7dc2162782333f5"
server_key_version: "encryptionkey@0"
I20260806 16:09:39.393787 28110 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:09:39.398643 28128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:39.403086 28110 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.006s sys 0.000s
I20260806 16:09:39.403343 28110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "4f247e66715a4fc6a7781a5e381ca821"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "3449c10e7c98720afd93c6a4ee768e0b"
server_key_iv: "886f34b0ccf72398b7dc2162782333f5"
server_key_version: "encryptionkey@0"
I20260806 16:09:39.403558 28110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:39.418751 28110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:39.422863 28110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:39.423125 28110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:39.423934 28110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:39.424954 28110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:39.425038 28110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:39.425113 28110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:39.425158 28110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:39.437810 28110 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:42337
I20260806 16:09:39.437860 28278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:42337 every 8 connection(s)
I20260806 16:09:39.439108 28110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:09:39.439862 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 28110
I20260806 16:09:39.440030 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:09:39.440544 8388 external_mini_cluster.cc:1428] Setting key 1e63eb2456b25820d7b9ec8ec45ca421
Aug 06 16:09:39 dist-test-slave-7z32 krb5kdc[27778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:09:39.462208 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.441591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:35575 (local address 127.8.49.62:37299)
0806 16:09:39.441844 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:39.441848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:39.442936 (+ 1088us) server_negotiation.cc:407] Connection header received
0806 16:09:39.445685 (+ 2749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:39.445689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:39.445750 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:39.445853 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:39.448000 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.448878 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.451314 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.451847 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.452094 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.456801 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:39.456805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:39.456807 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:39.456842 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:39.458492 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:39.459848 (+ 1356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:39.459854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:39.459856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:39.459996 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:39.460602 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:39.460608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:39.460613 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:39.460875 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:39.461010 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:39.461708 (+ 698us) server_negotiation.cc:299] Negotiation successful
0806 16:09:39.461892 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260806 16:09:39.463364 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.441821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:35575)
0806 16:09:39.442665 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:39.442769 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:39.445300 (+ 2531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:39.446199 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:39.446218 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:39.446875 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:39.447735 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.447765 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.449147 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.449151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.451023 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.451044 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.452243 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.452253 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.452665 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.453881 (+ 1216us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:39.453922 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:39.456424 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:39.458809 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:39.458885 (+ 76us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:39.458928 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:39.459593 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:39.460178 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:39.460182 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:39.460184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:39.460452 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:39.461026 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:39.461038 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:39.461391 (+ 353us) client_negotiation.cc:769] Sending connection context
0806 16:09:39.461638 (+ 247us) client_negotiation.cc:240] Negotiation successful
0806 16:09:39.462235 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":167,"threads_started":1}
I20260806 16:09:39.465118 28279 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37299
I20260806 16:09:39.465651 28279 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:39.466621 28279 heartbeater.cc:511] Master 127.8.49.62:37299 requested a full tablet report, sending...
I20260806 16:09:39.468465 27838 ts_manager.cc:194] Registered new tserver with Master: 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337)
I20260806 16:09:39.469448 27838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:35575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:09:39.488178 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37299
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:39.659327 28288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:39.659773 28288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:39.659837 28288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:39.665298 28288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:39.665419 28288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:39.665521 28288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:09:39.672803 28288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:37299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:39.674472 28288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:39.675973 28288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:39.689574 28300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:39.689641 28297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:39.689641 28298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:39.689857 28288 server_base.cc:1034] running on GCE node
I20260806 16:09:39.690761 28288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:39.691735 28288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:39.693034 28288 hybrid_clock.cc:648] HybridClock initialized: now 1786032579693008 us; error 62 us; skew 500 ppm
Aug 06 16:09:39 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:39.697599 28288 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:09:39.698992 28288 webserver.cc:460] Webserver started at http://127.8.49.3:41075/ using document root <none> and password file <none>
I20260806 16:09:39.699707 28288 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:39.699811 28288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:39.700075 28288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:09:39.702484 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "65c9bccde9a14df58a8ac83834ea7457"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "1272af24562827e32462f87fa327a660"
server_key_iv: "c3e9fb4e34bb0b8770f0d10f53022f1a"
server_key_version: "encryptionkey@0"
I20260806 16:09:39.703156 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "65c9bccde9a14df58a8ac83834ea7457"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "1272af24562827e32462f87fa327a660"
server_key_iv: "c3e9fb4e34bb0b8770f0d10f53022f1a"
server_key_version: "encryptionkey@0"
I20260806 16:09:39.708546 28288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 16:09:39.712545 28308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:39.714375 28288 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:09:39.714632 28288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "65c9bccde9a14df58a8ac83834ea7457"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "1272af24562827e32462f87fa327a660"
server_key_iv: "c3e9fb4e34bb0b8770f0d10f53022f1a"
server_key_version: "encryptionkey@0"
I20260806 16:09:39.714792 28288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:39.729895 28288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:39.734700 28288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:39.735056 28288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:39.735895 28288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:39.737038 28288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:39.737092 28288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:39.737175 28288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:39.737216 28288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:39.751313 28288 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:46297
I20260806 16:09:39.751356 28464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:46297 every 8 connection(s)
I20260806 16:09:39.752629 28288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:09:39.761619 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 28288
I20260806 16:09:39.761787 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:09:39.762149 8388 external_mini_cluster.cc:1428] Setting key 3858850e7c020dc90e48d255890d8c4a
Aug 06 16:09:39 dist-test-slave-7z32 krb5kdc[27778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:39.771668 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.755175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:44331 (local address 127.8.49.62:37299)
0806 16:09:39.755384 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:39.755390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:39.756910 (+ 1520us) server_negotiation.cc:407] Connection header received
0806 16:09:39.758162 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:39.758165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:39.758225 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:39.758342 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:39.760504 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.761120 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.762802 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.763145 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.763282 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.767411 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:39.767415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:39.767417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:39.767450 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:39.769032 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:39.769735 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:39.769738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:39.769740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:39.769829 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:39.770214 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:39.770216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:39.770217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:39.770416 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:39.770498 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:39.771351 (+ 853us) server_negotiation.cc:299] Negotiation successful
0806 16:09:39.771476 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 16:09:39.772922 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.755611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:44331)
0806 16:09:39.756668 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:39.756734 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:39.757881 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:39.758664 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:39.758673 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:39.759518 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:39.760294 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.760315 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.761343 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.761347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.762597 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.762610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.763470 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.763476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.763707 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.764786 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:39.764810 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:39.767178 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:39.769199 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:39.769213 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:39.769236 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:39.769606 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:39.769932 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:39.769935 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:39.769936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:39.770106 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:39.770554 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:39.770563 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:39.771036 (+ 473us) client_negotiation.cc:769] Sending connection context
0806 16:09:39.771266 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 16:09:39.771660 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":162,"threads_started":1}
I20260806 16:09:39.774489 28465 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37299
I20260806 16:09:39.774879 28465 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:39.775605 28465 heartbeater.cc:511] Master 127.8.49.62:37299 requested a full tablet report, sending...
I20260806 16:09:39.776996 27837 ts_manager.cc:194] Registered new tserver with Master: 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297)
I20260806 16:09:39.777783 27837 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:44331
I20260806 16:09:39.778156 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:09:39.790642 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.781306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.8.49.62:37299)
0806 16:09:39.781534 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:39.781538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:39.781637 (+ 99us) server_negotiation.cc:407] Connection header received
0806 16:09:39.781813 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:39.781816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:39.781865 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:39.781963 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:39.782956 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.783496 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.784616 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.784870 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.785001 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.786688 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:39.786691 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:39.786694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:39.786722 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:39.788348 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:39.788942 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:39.788946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:39.788948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:39.789080 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:39.789444 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:39.789447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:39.789448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:39.789766 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:39.789901 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:39.790325 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 16:09:39.790456 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 16:09:39.795442 27837 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:09:39.799574 27837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:09:39.823367 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.815169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37602 (local address 127.8.49.1:40719)
0806 16:09:39.815720 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:39.815733 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:39.815768 (+ 35us) server_negotiation.cc:407] Connection header received
0806 16:09:39.816143 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:39.816150 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:39.816456 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:39.816633 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:39.817174 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.818526 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.821442 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.822538 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.822708 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.823001 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:39.823072 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 16:09:39.823203 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":123,"threads_started":1}
I20260806 16:09:39.823378 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.814723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40719 (local address 127.0.0.1:37602)
0806 16:09:39.815591 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:39.815662 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:39.815869 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:39.816829 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:39.816832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:39.816850 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:39.817005 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.817014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.818702 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.818710 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.820916 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.820934 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.822705 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.822709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.822866 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.822893 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:09:39.822974 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 16:09:39.823160 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":123,"spinlock_wait_cycles":10496,"thread_start_us":208,"threads_started":1}
I20260806 16:09:39.823684 28492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.815600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:46297 (local address 127.0.0.1:56084)
0806 16:09:39.816055 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:39.816069 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:39.816179 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:39.817164 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:39.817167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:39.817185 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:39.817300 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.817307 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.818991 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.818995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.820917 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.820934 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.823163 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.823167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.823311 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.823342 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 16:09:39.823414 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 16:09:39.823518 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":367,"thread_start_us":156,"threads_started":1}
I20260806 16:09:39.823871 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.814572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48130 (local address 127.8.49.2:42337)
0806 16:09:39.815272 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:39.815286 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:39.815882 (+ 596us) server_negotiation.cc:407] Connection header received
0806 16:09:39.816140 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:39.816147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:39.816430 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:39.816616 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:39.816991 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.818623 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.820608 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.821506 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.821637 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.822221 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:39.823509 (+ 1288us) server_negotiation.cc:299] Negotiation successful
0806 16:09:39.823670 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":147,"threads_started":1}
I20260806 16:09:39.824185 28482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.814501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:42337 (local address 127.0.0.1:48130)
0806 16:09:39.815589 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:39.815658 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:39.815868 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:39.816627 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:39.816634 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:39.816663 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:39.816851 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.816859 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.818816 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.818843 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.820417 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.820431 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.822342 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.822347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.822697 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.822770 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 16:09:39.823127 (+ 357us) client_negotiation.cc:240] Negotiation successful
0806 16:09:39.823279 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":177,"spinlock_wait_cycles":2944,"thread_start_us":121,"threads_started":1}
I20260806 16:09:39.824734 28493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.816169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56084 (local address 127.8.49.3:46297)
0806 16:09:39.816667 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:39.816682 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:39.816708 (+ 26us) server_negotiation.cc:407] Connection header received
0806 16:09:39.816814 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:39.816819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:39.817000 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:39.817453 (+ 453us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:39.817664 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.818816 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.821153 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.822746 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.824015 (+ 1269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.824258 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:39.824335 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 16:09:39.824503 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":185,"threads_started":1}
I20260806 16:09:39.828037 28009 tablet_service.cc:1467] Processing CreateTablet for tablet ab829015a9504260bcb04d192b4355b4 (DEFAULT_TABLE table=test-table [id=0f8196556a5c4cf8b6088a5d1d058dee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:09:39.828550 28187 tablet_service.cc:1467] Processing CreateTablet for tablet ab829015a9504260bcb04d192b4355b4 (DEFAULT_TABLE table=test-table [id=0f8196556a5c4cf8b6088a5d1d058dee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:09:39.828547 28374 tablet_service.cc:1467] Processing CreateTablet for tablet ab829015a9504260bcb04d192b4355b4 (DEFAULT_TABLE table=test-table [id=0f8196556a5c4cf8b6088a5d1d058dee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:09:39.829464 28009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab829015a9504260bcb04d192b4355b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:39.830072 28374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab829015a9504260bcb04d192b4355b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:39.830066 28187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab829015a9504260bcb04d192b4355b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:39.839735 28496 tablet_bootstrap.cc:492] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Bootstrap starting.
I20260806 16:09:39.841233 28497 tablet_bootstrap.cc:492] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: Bootstrap starting.
I20260806 16:09:39.841454 28498 tablet_bootstrap.cc:492] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: Bootstrap starting.
I20260806 16:09:39.842918 28496 tablet_bootstrap.cc:654] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:39.843818 28496 log.cc:824] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:39.844705 28498 tablet_bootstrap.cc:654] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:39.845225 28497 tablet_bootstrap.cc:654] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:39.846655 28497 log.cc:824] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:39.846655 28498 log.cc:824] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:39.847688 28496 tablet_bootstrap.cc:492] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: No bootstrap required, opened a new log
I20260806 16:09:39.848002 28496 ts_tablet_manager.cc:1397] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 16:09:39.849781 28498 tablet_bootstrap.cc:492] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: No bootstrap required, opened a new log
I20260806 16:09:39.849781 28497 tablet_bootstrap.cc:492] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: No bootstrap required, opened a new log
I20260806 16:09:39.850087 28498 ts_tablet_manager.cc:1397] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 16:09:39.850087 28497 ts_tablet_manager.cc:1397] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 16:09:39.852880 28496 raft_consensus.cc:348] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:39.853274 28496 raft_consensus.cc:374] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:39.853343 28496 raft_consensus.cc:729] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e7a1e75b34142e18f120d73e01de516, State: Initialized, Role: FOLLOWER
I20260806 16:09:39.853948 28496 consensus_queue.cc:260] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:39.854929 28105 heartbeater.cc:503] Master 127.8.49.62:37299 was elected leader, sending a full tablet report...
I20260806 16:09:39.856453 28497 raft_consensus.cc:348] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:39.856453 28498 raft_consensus.cc:348] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:39.856930 28498 raft_consensus.cc:374] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:39.856930 28497 raft_consensus.cc:374] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:39.856930 28496 ts_tablet_manager.cc:1428] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260806 16:09:39.856995 28497 raft_consensus.cc:729] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f247e66715a4fc6a7781a5e381ca821, State: Initialized, Role: FOLLOWER
I20260806 16:09:39.857002 28498 raft_consensus.cc:729] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65c9bccde9a14df58a8ac83834ea7457, State: Initialized, Role: FOLLOWER
I20260806 16:09:39.857645 28498 consensus_queue.cc:260] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:39.857645 28497 consensus_queue.cc:260] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:39.858765 28279 heartbeater.cc:503] Master 127.8.49.62:37299 was elected leader, sending a full tablet report...
I20260806 16:09:39.859100 28497 ts_tablet_manager.cc:1428] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: Time spent starting tablet: real 0.009s user 0.007s sys 0.004s
I20260806 16:09:39.859289 28465 heartbeater.cc:503] Master 127.8.49.62:37299 was elected leader, sending a full tablet report...
I20260806 16:09:39.859604 28498 ts_tablet_manager.cc:1428] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: Time spent starting tablet: real 0.009s user 0.002s sys 0.008s
W20260806 16:09:39.895500 28106 tablet.cc:2369] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:09:39.941147 28280 tablet.cc:2369] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:09:39.941887 28503 raft_consensus.cc:482] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:09:39.942181 28503 raft_consensus.cc:504] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:39.943813 28503 leader_election.cc:288] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337), 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297)
I20260806 16:09:39.950667 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.944320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:46483 (local address 127.8.49.2:42337)
0806 16:09:39.944618 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:39.944626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:39.945084 (+ 458us) server_negotiation.cc:407] Connection header received
0806 16:09:39.945191 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:39.945194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:39.945292 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:39.945403 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:39.945872 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.946731 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.948961 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.949694 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.949849 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.949946 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:39.950387 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 16:09:39.950486 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 16:09:39.950728 28493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.944984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:47845 (local address 127.8.49.3:46297)
0806 16:09:39.945125 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:39.945131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:39.945262 (+ 131us) server_negotiation.cc:407] Connection header received
0806 16:09:39.945336 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:39.945342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:39.945404 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:39.945487 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:39.945872 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.946730 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.948961 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.949757 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.949866 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.949946 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:39.950387 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 16:09:39.950486 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 16:09:39.950712 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.944732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:46297 (local address 127.8.49.1:47845)
0806 16:09:39.945162 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:39.945174 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:39.945256 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:39.945523 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:39.945525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:39.945580 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:39.945703 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.945710 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.946930 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.946934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.948764 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.948778 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.949943 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.949947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.950165 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.950229 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 16:09:39.950341 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 16:09:39.950442 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":228,"spinlock_wait_cycles":43264,"thread_start_us":144,"threads_started":1}
I20260806 16:09:39.950712 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.944139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:42337 (local address 127.8.49.1:46483)
0806 16:09:39.944629 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:39.944680 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:39.944884 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:39.945440 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:39.945442 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:39.945483 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:39.945713 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.945720 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.946930 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.946934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.948764 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:39.948778 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.949943 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.949947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:39.950162 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.950235 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 16:09:39.950341 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 16:09:39.950442 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":331,"spinlock_wait_cycles":768,"thread_start_us":190,"threads_started":1}
I20260806 16:09:39.951851 28400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab829015a9504260bcb04d192b4355b4" candidate_uuid: "9e7a1e75b34142e18f120d73e01de516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c9bccde9a14df58a8ac83834ea7457" is_pre_election: true
I20260806 16:09:39.952100 28219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab829015a9504260bcb04d192b4355b4" candidate_uuid: "9e7a1e75b34142e18f120d73e01de516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f247e66715a4fc6a7781a5e381ca821" is_pre_election: true
I20260806 16:09:39.952365 28400 raft_consensus.cc:2454] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e7a1e75b34142e18f120d73e01de516 in term 0.
I20260806 16:09:39.952579 28219 raft_consensus.cc:2454] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e7a1e75b34142e18f120d73e01de516 in term 0.
I20260806 16:09:39.953195 27948 leader_election.cc:302] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65c9bccde9a14df58a8ac83834ea7457, 9e7a1e75b34142e18f120d73e01de516; no voters:
I20260806 16:09:39.953665 28503 raft_consensus.cc:2781] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:09:39.953809 28503 raft_consensus.cc:482] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:09:39.953859 28503 raft_consensus.cc:3037] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:39.955536 28503 raft_consensus.cc:504] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:39.956100 28503 leader_election.cc:288] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 1 election: Requested vote from peers 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337), 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297)
I20260806 16:09:39.956929 28400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab829015a9504260bcb04d192b4355b4" candidate_uuid: "9e7a1e75b34142e18f120d73e01de516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c9bccde9a14df58a8ac83834ea7457"
I20260806 16:09:39.957010 28219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab829015a9504260bcb04d192b4355b4" candidate_uuid: "9e7a1e75b34142e18f120d73e01de516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f247e66715a4fc6a7781a5e381ca821"
I20260806 16:09:39.957134 28400 raft_consensus.cc:3037] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:39.957216 28219 raft_consensus.cc:3037] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:39.958911 28219 raft_consensus.cc:2454] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e7a1e75b34142e18f120d73e01de516 in term 1.
I20260806 16:09:39.958930 28400 raft_consensus.cc:2454] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e7a1e75b34142e18f120d73e01de516 in term 1.
I20260806 16:09:39.959432 27948 leader_election.cc:302] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65c9bccde9a14df58a8ac83834ea7457, 9e7a1e75b34142e18f120d73e01de516; no voters:
I20260806 16:09:39.959733 28503 raft_consensus.cc:2781] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:09:39.960071 28503 raft_consensus.cc:686] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 1 LEADER]: Becoming Leader. State: Replica: 9e7a1e75b34142e18f120d73e01de516, State: Running, Role: LEADER
I20260806 16:09:39.960615 28503 consensus_queue.cc:237] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:39.965845 27838 catalog_manager.cc:5495] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e7a1e75b34142e18f120d73e01de516 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "9e7a1e75b34142e18f120d73e01de516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } health_report { overall_health: HEALTHY } } }
I20260806 16:09:39.986156 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:39.980497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37614 (local address 127.8.49.1:40719)
0806 16:09:39.980883 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:39.980888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:39.980908 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:09:39.981035 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:39.981038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:39.981158 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:39.981293 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:09:39.981807 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.982425 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.983844 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:39.984133 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:39.984262 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:39.984799 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:09:39.985477 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:09:39.985606 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:39.985802 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 16:09:39.985897 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260806 16:09:39.996517 28400 raft_consensus.cc:1261] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457 [term 1 FOLLOWER]: Refusing update from remote peer 9e7a1e75b34142e18f120d73e01de516: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:09:39.997002 28219 raft_consensus.cc:1261] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821 [term 1 FOLLOWER]: Refusing update from remote peer 9e7a1e75b34142e18f120d73e01de516: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:09:39.997881 28511 consensus_queue.cc:1035] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:09:39.998180 28503 consensus_queue.cc:1035] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 16:09:40.003993 28466 tablet.cc:2369] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:09:40.017400 28517 mvcc.cc:205] Tried to move back new op lower bound from 7315589447650664448 to 7315589447522390016. Current Snapshot: MvccSnapshot[applied={T|T < 7315589447650664448}]
I20260806 16:09:40.025859 27836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:09:40.026131 27836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:09:40.030082 27836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Sending DeleteTablet for 3 replicas of tablet ab829015a9504260bcb04d192b4355b4
I20260806 16:09:40.031530 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 27925
I20260806 16:09:40.031497 28009 tablet_service.cc:1514] Processing DeleteTablet for tablet ab829015a9504260bcb04d192b4355b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:09:40 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:37602
I20260806 16:09:40.031632 28187 tablet_service.cc:1514] Processing DeleteTablet for tablet ab829015a9504260bcb04d192b4355b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:09:40 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:48130
I20260806 16:09:40.045141 28374 tablet_service.cc:1514] Processing DeleteTablet for tablet ab829015a9504260bcb04d192b4355b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:09:40 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:56084
I20260806 16:09:40.045679 28527 tablet_replica.cc:330] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: stopping tablet replica
I20260806 16:09:40.046116 28529 tablet_replica.cc:330] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: stopping tablet replica
I20260806 16:09:40.046252 28527 raft_consensus.cc:2229] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:09:40.046430 28529 raft_consensus.cc:2229] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:09:40.046778 28529 raft_consensus.cc:2258] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:09:40.046779 28527 raft_consensus.cc:2258] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:09:40.049176 28527 ts_tablet_manager.cc:1905] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:09:40.049304 28529 ts_tablet_manager.cc:1905] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:09:40.053117 28527 ts_tablet_manager.cc:1918] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:09:40.053284 28527 log.cc:1196] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/ab829015a9504260bcb04d192b4355b4
I20260806 16:09:40.053705 28527 ts_tablet_manager.cc:1939] T ab829015a9504260bcb04d192b4355b4 P 4f247e66715a4fc6a7781a5e381ca821: Deleting consensus metadata
I20260806 16:09:40.054131 28529 ts_tablet_manager.cc:1918] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:09:40.054301 28529 log.cc:1196] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/ab829015a9504260bcb04d192b4355b4
I20260806 16:09:40.054685 28529 ts_tablet_manager.cc:1939] T ab829015a9504260bcb04d192b4355b4 P 65c9bccde9a14df58a8ac83834ea7457: Deleting consensus metadata
I20260806 16:09:40.055593 27820 catalog_manager.cc:4841] TS 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337): tablet ab829015a9504260bcb04d192b4355b4 (table test-table [id=0f8196556a5c4cf8b6088a5d1d058dee]) successfully deleted
W20260806 16:09:40.055990 27822 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:40719 (error 108)
I20260806 16:09:40.056476 27820 catalog_manager.cc:4841] TS 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297): tablet ab829015a9504260bcb04d192b4355b4 (table test-table [id=0f8196556a5c4cf8b6088a5d1d058dee]) successfully deleted
I20260806 16:09:40.057072 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 28110
I20260806 16:09:40.058141 28482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:40.057415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40719 (local address 127.0.0.1:37622)
0806 16:09:40.057861 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:40.057993 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268}
W20260806 16:09:40.058452 27822 catalog_manager.cc:4584] TS 9e7a1e75b34142e18f120d73e01de516 (127.8.49.1:40719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab829015a9504260bcb04d192b4355b4: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40719: connect: Connection refused (error 111)
I20260806 16:09:40.085654 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 28288
I20260806 16:09:40.101030 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 27799
I20260806 16:09:40.117255 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:37299
--webserver_interface=127.8.49.62
--webserver_port=40631
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:37299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:40.258534 28531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:40.258984 28531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:40.259044 28531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:40.263931 28531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:09:40.264022 28531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:40.264045 28531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:40.264065 28531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:09:40.264082 28531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:09:40.270462 28531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:37299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:37299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=40631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:40.271915 28531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:40.273154 28531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:40.293520 28541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:40.293715 28543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:40.293520 28539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:40.293977 28531 server_base.cc:1034] running on GCE node
I20260806 16:09:40.295116 28531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:40.296872 28531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:40.298171 28531 hybrid_clock.cc:648] HybridClock initialized: now 1786032580298133 us; error 54 us; skew 500 ppm
Aug 06 16:09:40 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:40.306351 28531 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:09:40.308267 28531 webserver.cc:460] Webserver started at http://127.8.49.62:40631/ using document root <none> and password file <none>
I20260806 16:09:40.309211 28531 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:40.309368 28531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:40.315380 28531 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:09:40.319031 28552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:40.321265 28531 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:09:40.321666 28531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "a103b579a9b34f27a98f92ab8f93c23e"
format_stamp: "Formatted at 2026-08-06 16:09:38 on dist-test-slave-7z32"
server_key: "7c9c83892bf8e542e7b9249d73a50371"
server_key_iv: "c08a78060b2e89c90a2a5d1de6a229e8"
server_key_version: "encryptionkey@0"
I20260806 16:09:40.322593 28531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:40.356415 28531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:40.361544 28531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:40.361898 28531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:40.373502 28531 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:37299
I20260806 16:09:40.373651 28644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:37299 every 8 connection(s)
I20260806 16:09:40.375187 28531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:09:40.378774 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 28531
I20260806 16:09:40.379411 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:40719
--local_ip_for_outbound_sockets=127.8.49.1
--tserver_master_addrs=127.8.49.62:37299
--webserver_port=45469
--webserver_interface=127.8.49.1
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:09:40.383184 28645 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:09:40.385869 28645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Bootstrap starting.
I20260806 16:09:40.408277 28645 log.cc:824] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:40.422329 28645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:09:40.422999 28645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Bootstrap complete.
I20260806 16:09:40.428741 28645 raft_consensus.cc:348] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:40.429291 28645 raft_consensus.cc:729] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a103b579a9b34f27a98f92ab8f93c23e, State: Initialized, Role: FOLLOWER
I20260806 16:09:40.429956 28645 consensus_queue.cc:260] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:40.430132 28645 raft_consensus.cc:388] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:09:40.430253 28645 raft_consensus.cc:482] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:09:40.430397 28645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:09:40.432996 28645 raft_consensus.cc:504] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:40.433439 28645 leader_election.cc:302] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a103b579a9b34f27a98f92ab8f93c23e; no voters:
I20260806 16:09:40.433892 28645 leader_election.cc:288] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 16:09:40.434015 28649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:09:40.434278 28649 raft_consensus.cc:686] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 2 LEADER]: Becoming Leader. State: Replica: a103b579a9b34f27a98f92ab8f93c23e, State: Running, Role: LEADER
I20260806 16:09:40.434787 28649 consensus_queue.cc:237] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:40.435611 28645 sys_catalog.cc:564] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:09:40.436802 28653 sys_catalog.cc:455] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a103b579a9b34f27a98f92ab8f93c23e. Latest consensus state: current_term: 2 leader_uuid: "a103b579a9b34f27a98f92ab8f93c23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } } }
I20260806 16:09:40.437029 28653 sys_catalog.cc:458] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:40.437520 28663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:09:40.437546 28652 sys_catalog.cc:455] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a103b579a9b34f27a98f92ab8f93c23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } } }
I20260806 16:09:40.437709 28652 sys_catalog.cc:458] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:40.443876 28663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:09:40.444638 28663 catalog_manager.cc:1259] Loaded cluster ID: 0372b8b360e544ca94522b722cd0c28d
I20260806 16:09:40.444886 28663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:09:40.446483 28663 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 16:09:40.452023 28663 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 16:09:40.459632 28663 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:09:40.460752 28663 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Loaded TSK: 0
I20260806 16:09:40.462301 28663 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:09:40.548692 28647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:40.549101 28647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:40.549153 28647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:40.554127 28647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:40.554239 28647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:40.554311 28647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:09:40.561336 28647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:40719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=45469
--tserver_master_addrs=127.8.49.62:37299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:40.563048 28647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:40.564673 28647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:40.577991 28684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:40.578009 28687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:40.578007 28681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:40.578642 28647 server_base.cc:1034] running on GCE node
I20260806 16:09:40.579566 28647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:40.580646 28647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:40.582041 28647 hybrid_clock.cc:648] HybridClock initialized: now 1786032580581981 us; error 70 us; skew 500 ppm
Aug 06 16:09:40 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:40.586301 28647 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:09:40.587926 28647 webserver.cc:460] Webserver started at http://127.8.49.1:45469/ using document root <none> and password file <none>
I20260806 16:09:40.588778 28647 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:40.588937 28647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:40.595049 28647 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:09:40.598091 28697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:40.599927 28647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:09:40.600071 28647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "9e7a1e75b34142e18f120d73e01de516"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "d5c1a125ee1a6a3ea67fbeaeab090f73"
server_key_iv: "adea759ac7f4400327a0580ca2eb4d82"
server_key_version: "encryptionkey@0"
I20260806 16:09:40.600875 28647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:40.625558 28647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:40.631855 28647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:40.632243 28647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:40.633179 28647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:40.635043 28707 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:09:40.640926 28647 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:09:40.641105 28647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260806 16:09:40.641265 28647 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:09:40.644533 28647 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:09:40.644670 28647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260806 16:09:40.645182 28707 tablet_bootstrap.cc:492] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Bootstrap starting.
I20260806 16:09:40.659539 28647 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:40719
I20260806 16:09:40.659682 28865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:40719 every 8 connection(s)
I20260806 16:09:40.661006 28647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:09:40.663281 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 28647
I20260806 16:09:40.663895 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:42337
--local_ip_for_outbound_sockets=127.8.49.2
--tserver_master_addrs=127.8.49.62:37299
--webserver_port=45921
--webserver_interface=127.8.49.2
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:09:40.666488 28707 log.cc:824] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:09:40 dist-test-slave-7z32 krb5kdc[27778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:40.678759 28707 tablet_bootstrap.cc:492] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:09:40.679404 28707 tablet_bootstrap.cc:492] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Bootstrap complete.
I20260806 16:09:40.681336 28707 ts_tablet_manager.cc:1397] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Time spent bootstrapping tablet: real 0.037s user 0.026s sys 0.008s
I20260806 16:09:40.685063 28707 raft_consensus.cc:348] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:40.685835 28707 raft_consensus.cc:729] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e7a1e75b34142e18f120d73e01de516, State: Initialized, Role: FOLLOWER
I20260806 16:09:40.686549 28707 consensus_queue.cc:260] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:40.687492 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:40.663939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:58625 (local address 127.8.49.62:37299)
0806 16:09:40.664615 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:40.664652 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:40.665257 (+ 605us) server_negotiation.cc:407] Connection header received
0806 16:09:40.667248 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:40.667266 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:40.667682 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:40.668133 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:40.669534 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.671212 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.673254 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.674269 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.674523 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:40.678994 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:40.679007 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:40.679039 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:40.679099 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:40.682260 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:40.683297 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:40.683305 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:40.683311 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:40.683489 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:40.684131 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:40.684134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:40.684136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:40.684744 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:40.684920 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:40.685897 (+ 977us) server_negotiation.cc:299] Negotiation successful
0806 16:09:40.686326 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":177,"threads_started":1}
I20260806 16:09:40.687760 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:40.663940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:58625)
0806 16:09:40.665004 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:40.665072 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:40.666464 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:40.668233 (+ 1769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:40.668243 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:40.668753 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:40.669291 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:40.669310 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.671460 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.671466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:40.673004 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:40.673025 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.674503 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.674507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:40.674757 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:40.675743 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:40.675773 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:40.678553 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:40.682550 (+ 3997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:40.682559 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:40.682579 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:40.683108 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:40.683683 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:40.683688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:40.683690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:40.683973 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:40.684976 (+ 1003us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:40.684983 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:40.685428 (+ 445us) client_negotiation.cc:769] Sending connection context
0806 16:09:40.685697 (+ 269us) client_negotiation.cc:240] Negotiation successful
0806 16:09:40.686457 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":193,"threads_started":1}
I20260806 16:09:40.688033 28707 ts_tablet_manager.cc:1428] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 16:09:40.691380 28866 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37299
I20260806 16:09:40.691778 28866 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:40.692512 28866 heartbeater.cc:511] Master 127.8.49.62:37299 requested a full tablet report, sending...
I20260806 16:09:40.696825 28580 ts_manager.cc:194] Registered new tserver with Master: 9e7a1e75b34142e18f120d73e01de516 (127.8.49.1:40719)
I20260806 16:09:40.702584 28580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:58625
I20260806 16:09:40.706647 28866 heartbeater.cc:503] Master 127.8.49.62:37299 was elected leader, sending a full tablet report...
Aug 06 16:09:40 dist-test-slave-7z32 krb5kdc[27778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:09:40.719882 28875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:40.701600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37634 (local address 127.8.49.1:40719)
0806 16:09:40.702494 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:40.702508 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:40.702546 (+ 38us) server_negotiation.cc:407] Connection header received
0806 16:09:40.702649 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:40.702656 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:40.703099 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:40.703287 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:40.704359 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.705912 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.707448 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.707971 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.708116 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:40.713489 (+ 5373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:40.713507 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:40.713525 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:40.713566 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:40.715869 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:40.717145 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:40.717153 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:40.717154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:40.717279 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:40.717689 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:40.717692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:40.717694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:40.718071 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:40.718249 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:40.719220 (+ 971us) server_negotiation.cc:299] Negotiation successful
0806 16:09:40.719507 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":728,"thread_start_us":211,"threads_started":1}
I20260806 16:09:40.719882 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:40.701438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40719 (local address 127.0.0.1:37634)
0806 16:09:40.702006 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:40.702077 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:40.702311 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:40.703346 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:40.703353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:40.703946 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:40.704148 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:40.704160 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.706131 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.706135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:40.707226 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:40.707247 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.708102 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.708131 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:40.708270 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:40.709044 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:40.709058 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:40.713167 (+ 4109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:40.716080 (+ 2913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:40.716100 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:40.716110 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:40.716507 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:40.717422 (+ 915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:40.717425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:40.717427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:40.717548 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:40.718218 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:40.718228 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:40.718417 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 16:09:40.718837 (+ 420us) client_negotiation.cc:240] Negotiation successful
0806 16:09:40.719516 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":131,"threads_started":1}
I20260806 16:09:40.721894 28772 tablet_service.cc:1514] Processing DeleteTablet for tablet ab829015a9504260bcb04d192b4355b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:09:40 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:37634
I20260806 16:09:40.723234 28876 tablet_replica.cc:330] stopping tablet replica
I20260806 16:09:40.723726 28876 raft_consensus.cc:2229] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:09:40.724108 28876 raft_consensus.cc:2258] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:09:40.725504 28876 ts_tablet_manager.cc:1905] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:09:40.731314 28876 ts_tablet_manager.cc:1918] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:09:40.731508 28876 log.cc:1196] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal/wals/ab829015a9504260bcb04d192b4355b4
I20260806 16:09:40.731844 28876 ts_tablet_manager.cc:1939] T ab829015a9504260bcb04d192b4355b4 P 9e7a1e75b34142e18f120d73e01de516: Deleting consensus metadata
I20260806 16:09:40.735797 28556 catalog_manager.cc:4841] TS 9e7a1e75b34142e18f120d73e01de516 (127.8.49.1:40719): tablet ab829015a9504260bcb04d192b4355b4 (table test-table [id=0f8196556a5c4cf8b6088a5d1d058dee]) successfully deleted
W20260806 16:09:40.816674 28870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:40.817029 28870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:40.817121 28870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:40.821695 28870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:40.821846 28870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:40.822144 28870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:09:40.829586 28870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:42337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=45921
--tserver_master_addrs=127.8.49.62:37299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:40.831051 28870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:40.832338 28870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:40.845758 28884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:40.845872 28888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:40.846066 28870 server_base.cc:1034] running on GCE node
W20260806 16:09:40.845698 28886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:40.847023 28870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:40.848268 28870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:40.849548 28870 hybrid_clock.cc:648] HybridClock initialized: now 1786032580849505 us; error 60 us; skew 500 ppm
Aug 06 16:09:40 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:40.853986 28870 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:09:40.855549 28870 webserver.cc:460] Webserver started at http://127.8.49.2:45921/ using document root <none> and password file <none>
I20260806 16:09:40.856364 28870 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:40.856511 28870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:40.862260 28870 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:09:40.865880 28898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:40.868244 28870 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:09:40.868487 28870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "4f247e66715a4fc6a7781a5e381ca821"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "3449c10e7c98720afd93c6a4ee768e0b"
server_key_iv: "886f34b0ccf72398b7dc2162782333f5"
server_key_version: "encryptionkey@0"
I20260806 16:09:40.869331 28870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:40.894431 28870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:40.900018 28870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:40.900334 28870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:40.901224 28870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:40.902452 28870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:40.902539 28870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:40.902629 28870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:40.902696 28870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:40.917949 28870 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:42337
I20260806 16:09:40.918077 29077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:42337 every 8 connection(s)
I20260806 16:09:40.919792 28870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:09:40.928617 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 28870
I20260806 16:09:40.929587 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:46297
--local_ip_for_outbound_sockets=127.8.49.3
--tserver_master_addrs=127.8.49.62:37299
--webserver_port=41075
--webserver_interface=127.8.49.3
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:09:40 dist-test-slave-7z32 krb5kdc[27778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:40.942477 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:40.923267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:51375 (local address 127.8.49.62:37299)
0806 16:09:40.923586 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:40.923591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:40.925326 (+ 1735us) server_negotiation.cc:407] Connection header received
0806 16:09:40.926450 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:40.926454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:40.926524 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:40.926622 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:40.928311 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.929555 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.931363 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.931919 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.932118 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:40.936795 (+ 4677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:40.936804 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:40.936812 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:40.936881 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:40.939300 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:40.940184 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:40.940194 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:40.940196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:40.940299 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:40.940801 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:40.940805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:40.940806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:40.941071 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:40.941195 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:40.941963 (+ 768us) server_negotiation.cc:299] Negotiation successful
0806 16:09:40.942169 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 16:09:40.944008 29081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:40.923894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:51375)
0806 16:09:40.925088 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:40.925159 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:40.926201 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:40.926905 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:40.926932 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:40.927390 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:40.927958 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:40.927973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.929892 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.929897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:40.931172 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:40.931187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:40.932297 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:40.932304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:40.932619 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:40.933685 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:40.933718 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:40.936404 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:40.939523 (+ 3119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:40.939534 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:40.939573 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:40.940024 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:40.940426 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:40.940430 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:40.940433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:40.940656 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:40.941216 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:40.941226 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:40.941630 (+ 404us) client_negotiation.cc:769] Sending connection context
0806 16:09:40.941898 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 16:09:40.942435 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":250,"threads_started":1}
I20260806 16:09:40.946084 29078 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37299
I20260806 16:09:40.946532 29078 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:40.947427 29078 heartbeater.cc:511] Master 127.8.49.62:37299 requested a full tablet report, sending...
I20260806 16:09:40.949206 28580 ts_manager.cc:194] Registered new tserver with Master: 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337)
I20260806 16:09:40.950098 28580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:51375
W20260806 16:09:41.084095 29082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:41.084522 29082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:41.084609 29082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:41.089401 29082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:41.089507 29082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:41.089628 29082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:09:41.096520 29082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:46297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=41075
--tserver_master_addrs=127.8.49.62:37299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:41.098076 29082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:41.099563 29082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:41.113349 29092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:41.113341 29095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:41.113492 29093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:41.113812 29082 server_base.cc:1034] running on GCE node
I20260806 16:09:41.114635 29082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:41.115912 29082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:41.119299 29082 hybrid_clock.cc:648] HybridClock initialized: now 1786032581119197 us; error 117 us; skew 500 ppm
Aug 06 16:09:41 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032581, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:41.123836 29082 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:09:41.125270 29082 webserver.cc:460] Webserver started at http://127.8.49.3:41075/ using document root <none> and password file <none>
I20260806 16:09:41.126085 29082 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:41.126187 29082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:41.132119 29082 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:09:41.136175 29103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:41.138501 29082 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:09:41.138713 29082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "65c9bccde9a14df58a8ac83834ea7457"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "1272af24562827e32462f87fa327a660"
server_key_iv: "c3e9fb4e34bb0b8770f0d10f53022f1a"
server_key_version: "encryptionkey@0"
I20260806 16:09:41.139528 29082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:41.181305 29082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:41.185982 29082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:41.186249 29082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:41.187131 29082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:41.188246 29082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:41.188308 29082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:41.188359 29082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:41.188452 29082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:41.202450 29082 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:46297
I20260806 16:09:41.202472 29256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:46297 every 8 connection(s)
I20260806 16:09:41.203997 29082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:09:41.204746 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 29082
I20260806 16:09:41.218602 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.205904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.8.49.62:37299)
0806 16:09:41.206099 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:41.206105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:41.206563 (+ 458us) server_negotiation.cc:407] Connection header received
0806 16:09:41.206818 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:41.206846 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:41.206921 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:41.207030 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:41.208359 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.209327 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.210787 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.211383 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.211547 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.213555 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:41.213559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:41.213561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:41.213602 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:41.215854 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:41.216551 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:41.216555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:41.216556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:41.216657 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:41.217193 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:41.217196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:41.217198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:41.217462 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:41.217725 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:41.218116 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 16:09:41.218321 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 16:09:41 dist-test-slave-7z32 krb5kdc[27778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032581, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:41.225363 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.206655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:56759 (local address 127.8.49.62:37299)
0806 16:09:41.206999 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:41.207006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:41.208419 (+ 1413us) server_negotiation.cc:407] Connection header received
0806 16:09:41.210603 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:41.210607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:41.210704 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:41.210879 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:41.213115 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.213900 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.216242 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.216659 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.216767 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.220721 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:41.220725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:41.220729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:41.220772 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:41.222682 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:41.223384 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:41.223393 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:41.223394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:41.223487 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:41.223930 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:41.223934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:41.223935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:41.224179 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:41.224299 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:41.224956 (+ 657us) server_negotiation.cc:299] Negotiation successful
0806 16:09:41.225133 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":97,"threads_started":1}
I20260806 16:09:41.226540 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.207326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:56759)
0806 16:09:41.208107 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:41.208185 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:41.210136 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:41.211322 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:41.211341 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:41.212055 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:41.212868 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:41.212887 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.214123 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.214127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:41.216073 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:41.216086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.216998 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.217002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:41.217310 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.218281 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:41.218308 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:41.220434 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:41.222869 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:41.222878 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:41.222905 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:41.223244 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:41.223606 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:41.223609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:41.223610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:41.223805 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:41.224299 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:41.224310 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:41.224665 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 16:09:41.224899 (+ 234us) client_negotiation.cc:240] Negotiation successful
0806 16:09:41.225371 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":198,"threads_started":1}
I20260806 16:09:41.228653 29257 heartbeater.cc:347] Connected to a master server at 127.8.49.62:37299
I20260806 16:09:41.229115 29257 heartbeater.cc:464] Registering TS with master...
I20260806 16:09:41.230108 29257 heartbeater.cc:511] Master 127.8.49.62:37299 requested a full tablet report, sending...
I20260806 16:09:41.232345 28580 ts_manager.cc:194] Registered new tserver with Master: 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297)
I20260806 16:09:41.233255 28580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:56759
I20260806 16:09:41.234905 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:09:41.242317 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.235937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.8.49.62:37299)
0806 16:09:41.236155 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:41.236161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:41.236531 (+ 370us) server_negotiation.cc:407] Connection header received
0806 16:09:41.236733 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:41.236735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:41.236989 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:41.237340 (+ 351us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:09:41.237759 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.238458 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.239944 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.240235 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.240352 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.240866 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:09:41.241491 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:09:41.241624 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:41.241969 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 16:09:41.242111 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 16:09:41.243603 28580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:09:41.247622 28580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:09:41.259294 28772 tablet_service.cc:1467] Processing CreateTablet for tablet 8fed31a088bc4f4690eb8751e37225ef (DEFAULT_TABLE table=test-table [id=d785634332c14f6d99d98797e4052d12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:09:41.260279 28772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fed31a088bc4f4690eb8751e37225ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:41.265300 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.258441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:46297 (local address 127.0.0.1:56092)
0806 16:09:41.259017 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:41.259034 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:41.259354 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:41.260277 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:41.260281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:41.260302 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:41.260515 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:41.260523 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.262449 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.262453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:41.263534 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:41.263544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.264706 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.264710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:41.264842 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.264895 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 16:09:41.265022 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 16:09:41.265117 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":1,"thread_start_us":200,"threads_started":1}
I20260806 16:09:41.265333 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.258730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.8.49.3:46297)
0806 16:09:41.259641 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:41.259661 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:41.259698 (+ 37us) server_negotiation.cc:407] Connection header received
0806 16:09:41.259813 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:41.259822 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:41.260082 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:41.260295 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:41.260672 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.262274 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.263706 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.264576 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.264703 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.264915 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:41.265065 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 16:09:41.265189 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":179,"threads_started":1}
I20260806 16:09:41.265774 29271 tablet_bootstrap.cc:492] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: Bootstrap starting.
I20260806 16:09:41.266364 29267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.257799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:42337 (local address 127.0.0.1:48132)
0806 16:09:41.258156 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:41.258179 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:41.258351 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:41.259834 (+ 1483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:41.259838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:41.259857 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:41.260008 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:41.260015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.261514 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.261516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:41.262980 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:41.262993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.265750 (+ 2757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.265754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:41.265906 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.265947 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 16:09:41.266054 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 16:09:41.266166 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":33,"thread_start_us":138,"threads_started":1}
I20260806 16:09:41.266671 29268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.258101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48132 (local address 127.8.49.2:42337)
0806 16:09:41.258731 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:41.258746 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:41.258778 (+ 32us) server_negotiation.cc:407] Connection header received
0806 16:09:41.259294 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:41.259302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:41.259609 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:41.259773 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:41.260140 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.261283 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.263179 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.264245 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.264389 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.265667 (+ 1278us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:41.266141 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 16:09:41.266304 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":215,"threads_started":1}
I20260806 16:09:41.267372 29271 tablet_bootstrap.cc:654] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:41.269281 29271 tablet_bootstrap.cc:492] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: No bootstrap required, opened a new log
I20260806 16:09:41.269441 29271 ts_tablet_manager.cc:1397] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:09:41.270005 29271 raft_consensus.cc:348] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:41.270165 29271 raft_consensus.cc:374] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:41.270210 29271 raft_consensus.cc:729] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e7a1e75b34142e18f120d73e01de516, State: Initialized, Role: FOLLOWER
I20260806 16:09:41.270368 29271 consensus_queue.cc:260] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:41.270774 29271 ts_tablet_manager.cc:1428] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:09:41.270632 28976 tablet_service.cc:1467] Processing CreateTablet for tablet 8fed31a088bc4f4690eb8751e37225ef (DEFAULT_TABLE table=test-table [id=d785634332c14f6d99d98797e4052d12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:09:41.270812 29165 tablet_service.cc:1467] Processing CreateTablet for tablet 8fed31a088bc4f4690eb8751e37225ef (DEFAULT_TABLE table=test-table [id=d785634332c14f6d99d98797e4052d12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:09:41.272272 28976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fed31a088bc4f4690eb8751e37225ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:41.272472 29165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fed31a088bc4f4690eb8751e37225ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:41.281656 29275 tablet_bootstrap.cc:492] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: Bootstrap starting.
I20260806 16:09:41.283411 29276 tablet_bootstrap.cc:492] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: Bootstrap starting.
I20260806 16:09:41.285615 29275 tablet_bootstrap.cc:654] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:41.286324 29276 tablet_bootstrap.cc:654] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:41.286896 29275 log.cc:824] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:41.287969 29276 log.cc:824] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:41.290164 29275 tablet_bootstrap.cc:492] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: No bootstrap required, opened a new log
I20260806 16:09:41.290473 29275 ts_tablet_manager.cc:1397] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 16:09:41.290848 29276 tablet_bootstrap.cc:492] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: No bootstrap required, opened a new log
I20260806 16:09:41.291118 29276 ts_tablet_manager.cc:1397] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260806 16:09:41.294406 29276 raft_consensus.cc:348] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:41.294809 29276 raft_consensus.cc:374] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:41.294919 29276 raft_consensus.cc:729] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65c9bccde9a14df58a8ac83834ea7457, State: Initialized, Role: FOLLOWER
I20260806 16:09:41.294601 29275 raft_consensus.cc:348] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:41.295063 29275 raft_consensus.cc:374] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:41.295115 29275 raft_consensus.cc:729] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f247e66715a4fc6a7781a5e381ca821, State: Initialized, Role: FOLLOWER
I20260806 16:09:41.295492 29276 consensus_queue.cc:260] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:41.295760 29275 consensus_queue.cc:260] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:41.296547 29257 heartbeater.cc:503] Master 127.8.49.62:37299 was elected leader, sending a full tablet report...
I20260806 16:09:41.296763 29078 heartbeater.cc:503] Master 127.8.49.62:37299 was elected leader, sending a full tablet report...
I20260806 16:09:41.296885 29276 ts_tablet_manager.cc:1428] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 16:09:41.297273 29275 ts_tablet_manager.cc:1428] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260806 16:09:41.413420 28867 tablet.cc:2369] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:09:41.421939 29079 tablet.cc:2369] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:09:41.455988 29258 tablet.cc:2369] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:09:41.611586 29274 raft_consensus.cc:482] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:09:41.611900 29274 raft_consensus.cc:504] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:41.613430 29274 leader_election.cc:288] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337), 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297)
I20260806 16:09:41.618544 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.614670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:41893 (local address 127.8.49.3:46297)
0806 16:09:41.614959 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:41.614965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:41.614984 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:09:41.615090 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:41.615093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:41.615153 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:41.615258 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:41.615632 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.616166 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.617429 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.617854 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.617989 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.618065 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:41.618296 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 16:09:41.618390 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 16:09:41.618762 29268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.614108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:48801 (local address 127.8.49.2:42337)
0806 16:09:41.614374 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:41.614383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:41.614417 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:09:41.614672 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:41.614676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:41.614750 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:41.615012 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:09:41.615398 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.615980 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.617471 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.617940 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.618076 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.618172 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:41.618439 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 16:09:41.618533 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 16:09:41.618556 29285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.614411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:46297 (local address 127.8.49.1:41893)
0806 16:09:41.614825 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:41.614861 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:41.614979 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:41.615278 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:41.615281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:41.615305 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:41.615501 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:41.615508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.616289 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.616293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:41.617291 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:41.617301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.617999 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.618003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:41.618099 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.618180 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 16:09:41.618277 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 16:09:41.618390 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":33,"thread_start_us":125,"threads_started":1}
I20260806 16:09:41.618981 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.613921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:42337 (local address 127.8.49.1:48801)
0806 16:09:41.614289 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:41.614309 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:41.614468 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:41.615046 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:41.615051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:41.615079 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:09:41.615251 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:41.615265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.616118 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.616123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:41.617320 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:41.617334 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.618095 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.618099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:41.618278 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.618301 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:09:41.618419 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 16:09:41.618529 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":77,"thread_start_us":131,"threads_started":1}
I20260806 16:09:41.620164 29194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fed31a088bc4f4690eb8751e37225ef" candidate_uuid: "9e7a1e75b34142e18f120d73e01de516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c9bccde9a14df58a8ac83834ea7457" is_pre_election: true
I20260806 16:09:41.620165 29005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fed31a088bc4f4690eb8751e37225ef" candidate_uuid: "9e7a1e75b34142e18f120d73e01de516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f247e66715a4fc6a7781a5e381ca821" is_pre_election: true
I20260806 16:09:41.620589 29194 raft_consensus.cc:2454] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e7a1e75b34142e18f120d73e01de516 in term 0.
I20260806 16:09:41.620589 29005 raft_consensus.cc:2454] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e7a1e75b34142e18f120d73e01de516 in term 0.
I20260806 16:09:41.621395 28699 leader_election.cc:302] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f247e66715a4fc6a7781a5e381ca821, 9e7a1e75b34142e18f120d73e01de516; no voters:
I20260806 16:09:41.621886 29274 raft_consensus.cc:2781] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:09:41.622018 29274 raft_consensus.cc:482] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:09:41.622072 29274 raft_consensus.cc:3037] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:41.623858 29274 raft_consensus.cc:504] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:41.624408 29274 leader_election.cc:288] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 1 election: Requested vote from peers 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337), 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297)
I20260806 16:09:41.625231 29005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fed31a088bc4f4690eb8751e37225ef" candidate_uuid: "9e7a1e75b34142e18f120d73e01de516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f247e66715a4fc6a7781a5e381ca821"
I20260806 16:09:41.625475 29005 raft_consensus.cc:3037] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:41.625797 29194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fed31a088bc4f4690eb8751e37225ef" candidate_uuid: "9e7a1e75b34142e18f120d73e01de516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c9bccde9a14df58a8ac83834ea7457"
I20260806 16:09:41.626013 29194 raft_consensus.cc:3037] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:41.627137 29005 raft_consensus.cc:2454] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e7a1e75b34142e18f120d73e01de516 in term 1.
I20260806 16:09:41.627691 28699 leader_election.cc:302] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f247e66715a4fc6a7781a5e381ca821, 9e7a1e75b34142e18f120d73e01de516; no voters:
I20260806 16:09:41.627851 29194 raft_consensus.cc:2454] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e7a1e75b34142e18f120d73e01de516 in term 1.
I20260806 16:09:41.627987 29274 raft_consensus.cc:2781] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:09:41.628376 29274 raft_consensus.cc:686] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 LEADER]: Becoming Leader. State: Replica: 9e7a1e75b34142e18f120d73e01de516, State: Running, Role: LEADER
I20260806 16:09:41.628736 29274 consensus_queue.cc:237] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:41.633294 28580 catalog_manager.cc:5495] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e7a1e75b34142e18f120d73e01de516 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "9e7a1e75b34142e18f120d73e01de516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } health_report { overall_health: HEALTHY } } }
I20260806 16:09:41.649966 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.645096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37640 (local address 127.8.49.1:40719)
0806 16:09:41.645560 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:41.645568 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:41.645583 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:09:41.645698 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:41.645701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:41.645770 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:41.645945 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:09:41.646260 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.646923 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.647848 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:41.648089 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:41.648210 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:41.648553 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:09:41.649207 (+ 654us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:09:41.649367 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:41.649408 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 16:09:41.649480 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":147,"threads_started":1}
I20260806 16:09:41.656317 29194 raft_consensus.cc:1261] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457 [term 1 FOLLOWER]: Refusing update from remote peer 9e7a1e75b34142e18f120d73e01de516: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:09:41.656306 29005 raft_consensus.cc:1261] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821 [term 1 FOLLOWER]: Refusing update from remote peer 9e7a1e75b34142e18f120d73e01de516: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:09:41.657599 29286 consensus_queue.cc:1035] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:09:41.657879 29274 consensus_queue.cc:1035] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:09:41.670310 29290 mvcc.cc:205] Tried to move back new op lower bound from 7315589454453137408 to 7315589454354841600. Current Snapshot: MvccSnapshot[applied={T|T < 7315589454453137408}]
I20260806 16:09:41.671132 29292 mvcc.cc:205] Tried to move back new op lower bound from 7315589454453137408 to 7315589454354841600. Current Snapshot: MvccSnapshot[applied={T|T < 7315589454453137408}]
I20260806 16:09:41.671795 29294 mvcc.cc:205] Tried to move back new op lower bound from 7315589454453137408 to 7315589454354841600. Current Snapshot: MvccSnapshot[applied={T|T < 7315589454453137408}]
I20260806 16:09:41.683976 28579 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:09:41.684254 28579 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:09:41.688283 28579 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Sending DeleteTablet for 3 replicas of tablet 8fed31a088bc4f4690eb8751e37225ef
I20260806 16:09:41.689520 28772 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fed31a088bc4f4690eb8751e37225ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:09:41 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:37634
I20260806 16:09:41.689742 29165 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fed31a088bc4f4690eb8751e37225ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:09:41 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:56092
I20260806 16:09:41.690325 29305 tablet_replica.cc:330] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: stopping tablet replica
I20260806 16:09:41.690327 28976 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fed31a088bc4f4690eb8751e37225ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:09:41 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:48132
I20260806 16:09:41.690675 29305 raft_consensus.cc:2229] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:09:41.691028 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 28647
I20260806 16:09:41.691056 29306 tablet_replica.cc:330] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: stopping tablet replica
I20260806 16:09:41.708024 29307 tablet_replica.cc:330] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: stopping tablet replica
I20260806 16:09:41.708287 29306 raft_consensus.cc:2229] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:09:41.708539 29307 raft_consensus.cc:2229] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:09:41.708879 29307 raft_consensus.cc:2258] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:09:41.708880 29306 raft_consensus.cc:2258] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:09:41.711403 29306 ts_tablet_manager.cc:1905] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:09:41.711403 29307 ts_tablet_manager.cc:1905] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:09:41.715734 29307 ts_tablet_manager.cc:1918] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:09:41.715734 29306 ts_tablet_manager.cc:1918] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:09:41.715916 29307 log.cc:1196] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/8fed31a088bc4f4690eb8751e37225ef
I20260806 16:09:41.715924 29306 log.cc:1196] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/8fed31a088bc4f4690eb8751e37225ef
I20260806 16:09:41.716374 29306 ts_tablet_manager.cc:1939] T 8fed31a088bc4f4690eb8751e37225ef P 65c9bccde9a14df58a8ac83834ea7457: Deleting consensus metadata
I20260806 16:09:41.716374 29307 ts_tablet_manager.cc:1939] T 8fed31a088bc4f4690eb8751e37225ef P 4f247e66715a4fc6a7781a5e381ca821: Deleting consensus metadata
W20260806 16:09:41.718101 28556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:40719 (error 108)
I20260806 16:09:41.718679 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 28870
I20260806 16:09:41.719383 28553 catalog_manager.cc:4841] TS 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297): tablet 8fed31a088bc4f4690eb8751e37225ef (table test-table [id=d785634332c14f6d99d98797e4052d12]) successfully deleted
I20260806 16:09:41.719642 28553 catalog_manager.cc:4841] TS 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337): tablet 8fed31a088bc4f4690eb8751e37225ef (table test-table [id=d785634332c14f6d99d98797e4052d12]) successfully deleted
I20260806 16:09:41.723208 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:41.722493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40719 (local address 127.0.0.1:37646)
0806 16:09:41.722997 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:41.723089 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393}
W20260806 16:09:41.725253 28556 catalog_manager.cc:4584] TS 9e7a1e75b34142e18f120d73e01de516 (127.8.49.1:40719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8fed31a088bc4f4690eb8751e37225ef: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40719: connect: Connection refused (error 111)
I20260806 16:09:41.734225 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 29082
I20260806 16:09:41.755841 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 28531
I20260806 16:09:41.774544 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:37299
--webserver_interface=127.8.49.62
--webserver_port=40631
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:37299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:41.934693 29309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:41.935216 29309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:41.935314 29309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:41.940039 29309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:09:41.940178 29309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:41.940215 29309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:41.940256 29309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:09:41.940296 29309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:09:41.947238 29309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:37299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:37299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=40631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:41.948951 29309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:41.950434 29309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:41.963407 29316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:41.963651 29315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:41.963789 29319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:41.963560 29309 server_base.cc:1034] running on GCE node
I20260806 16:09:41.964754 29309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:41.966239 29309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:41.969587 29309 hybrid_clock.cc:648] HybridClock initialized: now 1786032581969497 us; error 107 us; skew 500 ppm
Aug 06 16:09:41 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032581, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:41.973662 29309 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:09:41.975015 29309 webserver.cc:460] Webserver started at http://127.8.49.62:40631/ using document root <none> and password file <none>
I20260806 16:09:41.975713 29309 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:41.975812 29309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:41.980913 29309 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 16:09:41.984194 29329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:41.985958 29309 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 16:09:41.986308 29309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "a103b579a9b34f27a98f92ab8f93c23e"
format_stamp: "Formatted at 2026-08-06 16:09:38 on dist-test-slave-7z32"
server_key: "7c9c83892bf8e542e7b9249d73a50371"
server_key_iv: "c08a78060b2e89c90a2a5d1de6a229e8"
server_key_version: "encryptionkey@0"
I20260806 16:09:41.987074 29309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:42.022665 29309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:42.026446 29309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:42.026640 29309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:42.036427 29309 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:37299
I20260806 16:09:42.036950 29402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:37299 every 8 connection(s)
I20260806 16:09:42.037791 29309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:09:42.044332 29403 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:09:42.047009 29403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Bootstrap starting.
I20260806 16:09:42.047919 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 29309
I20260806 16:09:42.048683 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:40719
--local_ip_for_outbound_sockets=127.8.49.1
--tserver_master_addrs=127.8.49.62:37299
--webserver_port=45469
--webserver_interface=127.8.49.1
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:09:42.073662 29403 log.cc:824] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:42.087186 29403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:09:42.087657 29403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e: Bootstrap complete.
I20260806 16:09:42.092483 29403 raft_consensus.cc:348] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:42.092964 29403 raft_consensus.cc:729] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a103b579a9b34f27a98f92ab8f93c23e, State: Initialized, Role: FOLLOWER
I20260806 16:09:42.093505 29403 consensus_queue.cc:260] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:42.093657 29403 raft_consensus.cc:388] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:09:42.093753 29403 raft_consensus.cc:482] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:09:42.093881 29403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 2 FOLLOWER]: Advancing to term 3
I20260806 16:09:42.096266 29403 raft_consensus.cc:504] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:42.096647 29403 leader_election.cc:302] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a103b579a9b34f27a98f92ab8f93c23e; no voters:
I20260806 16:09:42.097060 29403 leader_election.cc:288] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 16:09:42.097163 29408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 3 FOLLOWER]: Leader election won for term 3
I20260806 16:09:42.097383 29408 raft_consensus.cc:686] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [term 3 LEADER]: Becoming Leader. State: Replica: a103b579a9b34f27a98f92ab8f93c23e, State: Running, Role: LEADER
I20260806 16:09:42.097723 29408 consensus_queue.cc:237] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } }
I20260806 16:09:42.098512 29403 sys_catalog.cc:564] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:09:42.098940 29408 sys_catalog.cc:455] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a103b579a9b34f27a98f92ab8f93c23e. Latest consensus state: current_term: 3 leader_uuid: "a103b579a9b34f27a98f92ab8f93c23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } } }
I20260806 16:09:42.099303 29408 sys_catalog.cc:458] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:42.100689 29416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:09:42.101472 29409 sys_catalog.cc:455] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a103b579a9b34f27a98f92ab8f93c23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a103b579a9b34f27a98f92ab8f93c23e" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 37299 } } }
I20260806 16:09:42.101615 29409 sys_catalog.cc:458] T 00000000000000000000000000000000 P a103b579a9b34f27a98f92ab8f93c23e [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:42.105557 29416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:09:42.106290 29416 catalog_manager.cc:1259] Loaded cluster ID: 0372b8b360e544ca94522b722cd0c28d
I20260806 16:09:42.106367 29416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 16:09:42.107425 29416 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbf661279ad google::LogMessage::Fail() at ??:0
@ 0x7fbf6612a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fbf661274de google::LogMessage::Flush() at ??:0
@ 0x7fbf66128a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbf70a6afdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fbf70a6bd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbf70a68579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fbf70aac640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fbf683793d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbf68379c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fbf6837b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fbf6836a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbf699ae6db start_thread at ??:0
@ 0x7fbf64c2561f clone at ??:0
W20260806 16:09:42.226969 29406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:42.227591 29406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:42.227844 29406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:42.233474 29406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:42.233654 29406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:42.233786 29406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:09:42.241482 29406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:40719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=45469
--tserver_master_addrs=127.8.49.62:37299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:42.243449 29406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:42.245411 29406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:42.260284 29458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:42.260253 29459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:42.260912 29462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:42.261078 29406 server_base.cc:1034] running on GCE node
I20260806 16:09:42.261837 29406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:42.262900 29406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:42.264323 29406 hybrid_clock.cc:648] HybridClock initialized: now 1786032582264276 us; error 62 us; skew 500 ppm
Aug 06 16:09:42 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032582, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:42.268564 29406 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:09:42.270148 29406 webserver.cc:460] Webserver started at http://127.8.49.1:45469/ using document root <none> and password file <none>
I20260806 16:09:42.271061 29406 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:42.271163 29406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:42.277390 29406 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:09:42.281914 29471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:42.286736 29406 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260806 16:09:42.287094 29406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "9e7a1e75b34142e18f120d73e01de516"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "d5c1a125ee1a6a3ea67fbeaeab090f73"
server_key_iv: "adea759ac7f4400327a0580ca2eb4d82"
server_key_version: "encryptionkey@0"
I20260806 16:09:42.288134 29406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:42.309676 29406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:42.315178 29406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:42.315431 29406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:42.316359 29406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:42.318173 29481 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:09:42.324676 29406 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:09:42.324833 29406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260806 16:09:42.324946 29406 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:09:42.328112 29406 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:09:42.328210 29406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260806 16:09:42.328827 29481 tablet_bootstrap.cc:492] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: Bootstrap starting.
I20260806 16:09:42.342950 29406 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:40719
I20260806 16:09:42.343268 29636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:40719 every 8 connection(s)
I20260806 16:09:42.344624 29406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:09:42.352129 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 29406
I20260806 16:09:42.352823 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:42337
--local_ip_for_outbound_sockets=127.8.49.2
--tserver_master_addrs=127.8.49.62:37299
--webserver_port=45921
--webserver_interface=127.8.49.2
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:09:42.363004 29481 log.cc:824] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:42.379199 29481 tablet_bootstrap.cc:492] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:09:42.379854 29481 tablet_bootstrap.cc:492] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: Bootstrap complete.
I20260806 16:09:42.381783 29481 ts_tablet_manager.cc:1397] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: Time spent bootstrapping tablet: real 0.053s user 0.043s sys 0.004s
W20260806 16:09:42.385931 29643 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:09:42.348042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:42483)
0806 16:09:42.348926 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:42.349020 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:42.350707 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:42.384316 (+ 33609us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":217,"threads_started":1}
I20260806 16:09:42.385689 29481 raft_consensus.cc:348] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:42.386261 29481 raft_consensus.cc:729] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e7a1e75b34142e18f120d73e01de516, State: Initialized, Role: FOLLOWER
I20260806 16:09:42.386935 29481 consensus_queue.cc:260] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
W20260806 16:09:42.387292 29637 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:37299 (0 consecutive failures): Network error: Failed to ping master at 127.8.49.62:37299: Client connection negotiation failed: client connection to 127.8.49.62:37299: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 16:09:42.388154 29481 ts_tablet_manager.cc:1428] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260806 16:09:42.388572 29643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:42.388136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:48857)
0806 16:09:42.388344 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:42.388462 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 16:09:42.389566 29643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:42.389285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:37711)
0806 16:09:42.389428 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:42.389490 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 16:09:42.389943 29637 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:09:42.533651 29644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:42.534121 29644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:42.534214 29644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:42.541441 29644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:42.541601 29644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:42.541728 29644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:09:42.550179 29644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:42337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=45921
--tserver_master_addrs=127.8.49.62:37299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:42.552090 29644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:42.553685 29644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:42.568576 29656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:42.568714 29644 server_base.cc:1034] running on GCE node
W20260806 16:09:42.568688 29655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:42.568624 29659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:42.569475 29644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:42.570379 29644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:42.571625 29644 hybrid_clock.cc:648] HybridClock initialized: now 1786032582571569 us; error 79 us; skew 500 ppm
Aug 06 16:09:42 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032582, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:42.575999 29644 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:09:42.577229 29644 webserver.cc:460] Webserver started at http://127.8.49.2:45921/ using document root <none> and password file <none>
I20260806 16:09:42.578004 29644 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:42.578114 29644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:42.583840 29644 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:09:42.587586 29671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:42.589905 29644 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:09:42.590148 29644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "4f247e66715a4fc6a7781a5e381ca821"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "3449c10e7c98720afd93c6a4ee768e0b"
server_key_iv: "886f34b0ccf72398b7dc2162782333f5"
server_key_version: "encryptionkey@0"
I20260806 16:09:42.591030 29644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:42.619834 29644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:42.624596 29644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:42.625028 29644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:42.625921 29644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:42.627162 29644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:42.627245 29644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:42.627322 29644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:42.627369 29644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:42.640092 29644 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:42337
I20260806 16:09:42.640213 29836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:42337 every 8 connection(s)
I20260806 16:09:42.641419 29644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:09:42.644433 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 29644
I20260806 16:09:42.645164 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:46297
--local_ip_for_outbound_sockets=127.8.49.3
--tserver_master_addrs=127.8.49.62:37299
--webserver_port=41075
--webserver_interface=127.8.49.3
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:09:42.647182 29841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:42.644893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:43681)
0806 16:09:42.645508 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:42.646251 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":173,"threads_started":1}
W20260806 16:09:42.648169 29838 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:37299 (0 consecutive failures): Network error: Failed to ping master at 127.8.49.62:37299: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
I20260806 16:09:42.649307 29841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:42.648913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:56567)
0806 16:09:42.649119 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:42.649202 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 16:09:42.650370 29841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:42.650007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:60175)
0806 16:09:42.650219 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:42.650287 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260806 16:09:42.650686 29838 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:09:42.782965 29842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:42.783320 29842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:42.783406 29842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:42.787776 29842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:42.787887 29842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:42.787994 29842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:09:42.793938 29842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:46297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=41075
--tserver_master_addrs=127.8.49.62:37299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:42.795358 29842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:42.796658 29842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:42.809639 29852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:42.809746 29849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:42.809660 29848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:42.809940 29842 server_base.cc:1034] running on GCE node
I20260806 16:09:42.810681 29842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:42.811786 29842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:42.813058 29842 hybrid_clock.cc:648] HybridClock initialized: now 1786032582813007 us; error 72 us; skew 500 ppm
Aug 06 16:09:42 dist-test-slave-7z32 krb5kdc[27778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032582, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:42.817703 29842 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:09:42.819314 29842 webserver.cc:460] Webserver started at http://127.8.49.3:41075/ using document root <none> and password file <none>
I20260806 16:09:42.820245 29842 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:42.820370 29842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:42.826143 29842 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 16:09:42.829541 29859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:42.831883 29842 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 16:09:42.832145 29842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "65c9bccde9a14df58a8ac83834ea7457"
format_stamp: "Formatted at 2026-08-06 16:09:39 on dist-test-slave-7z32"
server_key: "1272af24562827e32462f87fa327a660"
server_key_iv: "c3e9fb4e34bb0b8770f0d10f53022f1a"
server_key_version: "encryptionkey@0"
I20260806 16:09:42.833002 29842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:42.859404 29842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:42.865366 29842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:42.865679 29842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:42.866593 29842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:09:42.867887 29842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:09:42.867977 29842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:42.868065 29842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:09:42.868111 29842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:42.882159 29842 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:46297
I20260806 16:09:42.882222 30013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:46297 every 8 connection(s)
I20260806 16:09:42.883776 29842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:09:42.888649 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 29842
I20260806 16:09:42.889245 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:42.886665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:58537)
0806 16:09:42.887809 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:42.887980 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":821,"thread_start_us":245,"threads_started":1}
W20260806 16:09:42.890190 30015 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:37299 (0 consecutive failures): Network error: Failed to ping master at 127.8.49.62:37299: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
I20260806 16:09:42.891261 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:42.890696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:46919)
0806 16:09:42.891068 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:42.891129 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 16:09:42.892144 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:42.891880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:44739)
0806 16:09:42.892017 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:42.892072 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 16:09:42.892606 30015 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 16:09:43.391734 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:43.390937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:57145)
0806 16:09:43.391388 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:43.391582 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":114,"threads_started":1}
I20260806 16:09:43.539225 30024 raft_consensus.cc:482] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:09:43.539579 30024 raft_consensus.cc:504] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:43.541354 30024 leader_election.cc:288] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337), 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297)
Aug 06 16:09:43 dist-test-slave-7z32 krb5kdc[27778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032582, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.2@KRBTEST.COM
Aug 06 16:09:43 dist-test-slave-7z32 krb5kdc[27778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032582, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.3@KRBTEST.COM
I20260806 16:09:43.563498 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:43.542275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:46297 (local address 127.8.49.1:46797)
0806 16:09:43.542762 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:43.542791 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:43.542960 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:43.545002 (+ 2042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:43.545006 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:43.545397 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:43.546098 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:43.546119 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:43.548420 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:43.548424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:43.549377 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:43.549402 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:43.550537 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:43.550542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:43.550630 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:43.551531 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:43.551560 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:43.554297 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:43.558250 (+ 3953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:43.558262 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:43.558291 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:43.559363 (+ 1072us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:43.560053 (+ 690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:43.560057 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:43.560058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:43.560357 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:43.561624 (+ 1267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:43.561633 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:43.562252 (+ 619us) client_negotiation.cc:769] Sending connection context
0806 16:09:43.562561 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 16:09:43.563198 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":362,"spinlock_wait_cycles":32000,"thread_start_us":107,"threads_started":1}
I20260806 16:09:43.563591 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:43.542803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:46797 (local address 127.8.49.3:46297)
0806 16:09:43.543288 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:43.543303 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:43.543348 (+ 45us) server_negotiation.cc:407] Connection header received
0806 16:09:43.544129 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:43.544136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:43.544644 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:43.545003 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:43.546400 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:43.548184 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:43.549583 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:43.550353 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:43.550549 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:43.554582 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:43.554588 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:43.554601 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:43.554652 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:43.557960 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:43.559596 (+ 1636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:43.559607 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:43.559616 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:43.559878 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:43.560542 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:43.560546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:43.560548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:43.561412 (+ 864us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:43.561600 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:43.562800 (+ 1200us) server_negotiation.cc:299] Negotiation successful
0806 16:09:43.563377 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":143,"threads_started":1}
I20260806 16:09:43.563591 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:43.542207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:40047 (local address 127.8.49.2:42337)
0806 16:09:43.542859 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:43.542884 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:43.542947 (+ 63us) server_negotiation.cc:407] Connection header received
0806 16:09:43.543793 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:43.543801 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:43.544242 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:43.544565 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:43.546344 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:43.547512 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:43.549103 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:43.549928 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:43.550118 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:43.554471 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:43.554482 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:43.554507 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:43.554562 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:43.557961 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:43.559596 (+ 1635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:43.559607 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:43.559616 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:43.559878 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:43.560526 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:43.560530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:43.560532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:43.561412 (+ 880us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:43.561600 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:43.562800 (+ 1200us) server_negotiation.cc:299] Negotiation successful
0806 16:09:43.563334 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":114,"threads_started":1}
I20260806 16:09:43.563525 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:43.541863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:42337 (local address 127.8.49.1:40047)
0806 16:09:43.542095 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:43.542119 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 16:09:43.542384 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:09:43.544699 (+ 2315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:09:43.544706 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:09:43.545384 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:09:43.546097 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:43.546117 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:43.547700 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:43.547704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:43.548921 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:09:43.548934 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:09:43.550155 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:09:43.550160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:09:43.550440 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:43.551579 (+ 1139us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:09:43.551595 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:09:43.554014 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:09:43.558250 (+ 4236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:43.558262 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:43.558291 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:43.559363 (+ 1072us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:43.560059 (+ 696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:09:43.560061 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:09:43.560063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:09:43.560357 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:09:43.561624 (+ 1267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:09:43.561633 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:09:43.562252 (+ 619us) client_negotiation.cc:769] Sending connection context
0806 16:09:43.562561 (+ 309us) client_negotiation.cc:240] Negotiation successful
0806 16:09:43.563124 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":287,"spinlock_wait_cycles":38784}
W20260806 16:09:43.567898 29473 leader_election.cc:341] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297): Not found: Tablet not found: 8fed31a088bc4f4690eb8751e37225ef
W20260806 16:09:43.568960 29473 leader_election.cc:341] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337): Not found: Tablet not found: 8fed31a088bc4f4690eb8751e37225ef
I20260806 16:09:43.569203 29473 leader_election.cc:302] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e7a1e75b34142e18f120d73e01de516; no voters: 4f247e66715a4fc6a7781a5e381ca821, 65c9bccde9a14df58a8ac83834ea7457
I20260806 16:09:43.569664 30024 raft_consensus.cc:2726] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:09:43.652534 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:43.651701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:45819)
0806 16:09:43.652298 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:43.652377 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":136,"threads_started":1}
I20260806 16:09:43.894533 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:43.893693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:57771)
0806 16:09:43.894253 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:43.894352 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":122,"threads_started":1}
I20260806 16:09:44.393836 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:44.393027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:58413)
0806 16:09:44.393499 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:44.393659 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":100,"threads_started":1}
I20260806 16:09:44.654636 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:44.653922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:53287)
0806 16:09:44.654350 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:44.654447 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":108,"threads_started":1}
I20260806 16:09:44.896632 30038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:44.895983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:56663)
0806 16:09:44.896364 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:44.896484 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":109,"threads_started":1}
I20260806 16:09:45.395526 30039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:45.394984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:48093)
0806 16:09:45.395330 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:45.395407 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":133,"threads_started":1}
I20260806 16:09:45.656714 30040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:45.655903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:54867)
0806 16:09:45.656443 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:45.656564 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":107,"threads_started":1}
I20260806 16:09:45.787029 30041 raft_consensus.cc:482] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:09:45.787212 30041 raft_consensus.cc:504] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:45.787843 30041 leader_election.cc:288] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337), 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297)
W20260806 16:09:45.789597 29473 leader_election.cc:341] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297): Not found: Tablet not found: 8fed31a088bc4f4690eb8751e37225ef
W20260806 16:09:45.789870 29473 leader_election.cc:341] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337): Not found: Tablet not found: 8fed31a088bc4f4690eb8751e37225ef
I20260806 16:09:45.789950 29473 leader_election.cc:302] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e7a1e75b34142e18f120d73e01de516; no voters: 4f247e66715a4fc6a7781a5e381ca821, 65c9bccde9a14df58a8ac83834ea7457
I20260806 16:09:45.790225 30041 raft_consensus.cc:2726] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:09:45.898504 30043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:45.897853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:51327)
0806 16:09:45.898309 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:45.898387 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":153,"threads_started":1}
I20260806 16:09:46.397704 30044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:46.396861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:39709)
0806 16:09:46.397356 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:46.397517 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":118,"threads_started":1}
I20260806 16:09:46.658704 30045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:46.657969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:52811)
0806 16:09:46.658462 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:46.658563 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":137,"threads_started":1}
I20260806 16:09:46.900594 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:46.899871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:43727)
0806 16:09:46.900368 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:46.900468 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":118,"threads_started":1}
I20260806 16:09:47.399889 30047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:47.399232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:33149)
0806 16:09:47.399688 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:47.399776 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":110,"threads_started":1}
I20260806 16:09:47.660990 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:47.660250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:60321)
0806 16:09:47.660723 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:47.660816 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":128,"threads_started":1}
I20260806 16:09:47.835179 30049 raft_consensus.cc:482] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:09:47.835397 30049 raft_consensus.cc:504] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:47.836002 30049 leader_election.cc:288] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337), 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297)
W20260806 16:09:47.837359 29473 leader_election.cc:341] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297): Not found: Tablet not found: 8fed31a088bc4f4690eb8751e37225ef
W20260806 16:09:47.837505 29473 leader_election.cc:341] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337): Not found: Tablet not found: 8fed31a088bc4f4690eb8751e37225ef
I20260806 16:09:47.837548 29473 leader_election.cc:302] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e7a1e75b34142e18f120d73e01de516; no voters: 4f247e66715a4fc6a7781a5e381ca821, 65c9bccde9a14df58a8ac83834ea7457
I20260806 16:09:47.837864 30049 raft_consensus.cc:2726] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:09:47.902541 30051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:47.901771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:46449)
0806 16:09:47.902298 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:47.902399 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":148,"threads_started":1}
I20260806 16:09:48.401640 30052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:48.401030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:49547)
0806 16:09:48.401423 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:48.401521 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":119,"threads_started":1}
I20260806 16:09:48.663111 30053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:48.662331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:55903)
0806 16:09:48.662887 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:48.662987 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":143,"threads_started":1}
I20260806 16:09:48.904469 30054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:48.903833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:58299)
0806 16:09:48.904225 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:48.904325 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":119,"threads_started":1}
I20260806 16:09:49.403668 30055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:49.402975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:47173)
0806 16:09:49.403410 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:49.403536 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":132,"threads_started":1}
I20260806 16:09:49.664922 30056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:49.664097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:56587)
0806 16:09:49.664649 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:49.664768 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":94,"threads_started":1}
I20260806 16:09:49.906915 30057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:49.905957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:51051)
0806 16:09:49.906606 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:49.906704 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":174,"threads_started":1}
I20260806 16:09:50.405720 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:50.405024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:51041)
0806 16:09:50.405496 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:50.405595 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":133,"threads_started":1}
I20260806 16:09:50.666728 30059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:50.666108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:36015)
0806 16:09:50.666512 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:50.666605 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":134,"threads_started":1}
I20260806 16:09:50.774194 30060 raft_consensus.cc:482] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:09:50.774374 30060 raft_consensus.cc:504] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:50.774969 30060 leader_election.cc:288] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337), 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297)
W20260806 16:09:50.776439 29473 leader_election.cc:341] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297): Not found: Tablet not found: 8fed31a088bc4f4690eb8751e37225ef
W20260806 16:09:50.776618 29473 leader_election.cc:341] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337): Not found: Tablet not found: 8fed31a088bc4f4690eb8751e37225ef
I20260806 16:09:50.776760 29473 leader_election.cc:302] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e7a1e75b34142e18f120d73e01de516; no voters: 4f247e66715a4fc6a7781a5e381ca821, 65c9bccde9a14df58a8ac83834ea7457
I20260806 16:09:50.777038 30060 raft_consensus.cc:2726] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:09:50.908898 30061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:50.908174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:40099)
0806 16:09:50.908681 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:50.908774 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":158,"threads_started":1}
I20260806 16:09:51.407662 30062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:51.406984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:34835)
0806 16:09:51.407417 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:51.407516 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":126,"threads_started":1}
I20260806 16:09:51.668762 30063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:51.668058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:59903)
0806 16:09:51.668513 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:51.668613 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":125,"threads_started":1}
I20260806 16:09:51.911173 30064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:51.910343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:54917)
0806 16:09:51.910752 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:51.910989 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":114,"threads_started":1}
I20260806 16:09:52.409902 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:52.409108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:48123)
0806 16:09:52.409597 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:52.409717 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":118,"threads_started":1}
I20260806 16:09:52.671182 30067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:52.670205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:53429)
0806 16:09:52.670870 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:52.671001 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":105,"threads_started":1}
I20260806 16:09:52.913116 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:52.912451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:42989)
0806 16:09:52.912876 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:52.912975 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":122,"threads_started":1}
I20260806 16:09:53.412159 30069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:53.411399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:37111)
0806 16:09:53.411845 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:53.411965 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":107,"threads_started":1}
I20260806 16:09:53.673858 30070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:53.672908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:52341)
0806 16:09:53.673526 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:53.673697 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":183,"threads_started":1}
I20260806 16:09:53.915246 30071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:53.914352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:49755)
0806 16:09:53.914757 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:53.915059 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":101,"threads_started":1}
I20260806 16:09:54.414361 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:54.413818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:55115)
0806 16:09:54.414169 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:54.414247 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":117,"threads_started":1}
I20260806 16:09:54.573959 30073 raft_consensus.cc:482] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:09:54.574177 30073 raft_consensus.cc:504] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f247e66715a4fc6a7781a5e381ca821" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 42337 } } peers { permanent_uuid: "65c9bccde9a14df58a8ac83834ea7457" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46297 } } peers { permanent_uuid: "9e7a1e75b34142e18f120d73e01de516" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40719 } }
I20260806 16:09:54.574707 30073 leader_election.cc:288] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337), 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297)
W20260806 16:09:54.576413 29473 leader_election.cc:341] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 65c9bccde9a14df58a8ac83834ea7457 (127.8.49.3:46297): Not found: Tablet not found: 8fed31a088bc4f4690eb8751e37225ef
W20260806 16:09:54.576623 29473 leader_election.cc:341] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4f247e66715a4fc6a7781a5e381ca821 (127.8.49.2:42337): Not found: Tablet not found: 8fed31a088bc4f4690eb8751e37225ef
I20260806 16:09:54.576663 29473 leader_election.cc:302] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e7a1e75b34142e18f120d73e01de516; no voters: 4f247e66715a4fc6a7781a5e381ca821, 65c9bccde9a14df58a8ac83834ea7457
I20260806 16:09:54.576901 30073 raft_consensus.cc:2726] T 8fed31a088bc4f4690eb8751e37225ef P 9e7a1e75b34142e18f120d73e01de516 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:09:54.676167 30074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:54.675379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:55599)
0806 16:09:54.675892 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:54.675999 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":138,"threads_started":1}
I20260806 16:09:54.917161 30075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:54.916423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:41973)
0806 16:09:54.916891 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:54.917004 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":96,"threads_started":1}
I20260806 16:09:55.416338 30076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:55.415643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:51791)
0806 16:09:55.416127 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:55.416219 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":131,"threads_started":1}
I20260806 16:09:55.677940 30079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:55.677271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:47093)
0806 16:09:55.677713 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:55.677799 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":149,"threads_started":1}
I20260806 16:09:55.919356 30080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:55.918627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:46229)
0806 16:09:55.919131 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:55.919233 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":188,"threads_started":1}
I20260806 16:09:56.418774 30081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:56.418056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:43889)
0806 16:09:56.418489 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:56.418595 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":108,"threads_started":1}
I20260806 16:09:56.679689 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:56.679011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:39417)
0806 16:09:56.679477 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:56.679556 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":122,"threads_started":1}
I20260806 16:09:56.921502 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:56.920827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:43361)
0806 16:09:56.921272 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:56.921369 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":146,"threads_started":1}
I20260806 16:09:57.420917 30086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:57.420288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.1:54403)
0806 16:09:57.420715 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:57.420794 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":134,"threads_started":1}
I20260806 16:09:57.682015 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:57.681196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.2:57373)
0806 16:09:57.681722 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:57.681835 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":92,"threads_started":1}
I20260806 16:09:57.890313 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 29406
I20260806 16:09:57.914605 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 29644
I20260806 16:09:57.934106 30088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:57.931950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:37299 (local address 127.8.49.3:49431)
0806 16:09:57.933798 (+ 1848us) negotiation.cc:107] Waiting for socket to connect
0806 16:09:57.933925 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1671,"thread_start_us":1442,"threads_started":1}
I20260806 16:09:57.941584 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 29842
2026-08-06T16:09:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20985 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:09:58 dist-test-slave-7z32 krb5kdc[30092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:09:58 dist-test-slave-7z32 krb5kdc[30092](info): set up 2 sockets
Aug 06 16:09:58 dist-test-slave-7z32 krb5kdc[30092](info): commencing operation
krb5kdc: starting...
W20260806 16:09:59.357245 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.381s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:09:59 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:09:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:09:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:09:59.540786 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:44761
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:44761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:09:59.706209 30110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:09:59.706776 30110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:09:59.706923 30110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:09:59.712100 30110 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 16:09:59.712239 30110 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 16:09:59.712270 30110 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 16:09:59.712294 30110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:09:59.712342 30110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:09:59.712389 30110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:09:59.712450 30110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:09:59.712489 30110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:09:59.720175 30110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:44761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:44761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:09:59.722049 30110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:09:59.723568 30110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:09:59.736155 30118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:09:59.736164 30115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:59.736299 30110 server_base.cc:1034] running on GCE node
W20260806 16:09:59.736205 30116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:09:59.737459 30110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:09:59.739113 30110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:09:59.740362 30110 hybrid_clock.cc:648] HybridClock initialized: now 1786032599740329 us; error 61 us; skew 500 ppm
Aug 06 16:09:59 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032599, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:09:59.750178 30110 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:09:59.751706 30110 webserver.cc:460] Webserver started at http://127.8.49.62:40337/ using document root <none> and password file <none>
I20260806 16:09:59.752543 30110 fs_manager.cc:359] Metadata directory not provided
I20260806 16:09:59.752661 30110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:09:59.752972 30110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:09:59.755823 30110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "0f5373aaab0349988cf7a85813c5b055"
format_stamp: "Formatted at 2026-08-06 16:09:59 on dist-test-slave-7z32"
server_key: "0a9fa1b3a7abb0ad9e017b400dc6749f"
server_key_iv: "2aa50cfb525141960a2e94215a6c53fe"
server_key_version: "encryptionkey@0"
I20260806 16:09:59.756582 30110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "0f5373aaab0349988cf7a85813c5b055"
format_stamp: "Formatted at 2026-08-06 16:09:59 on dist-test-slave-7z32"
server_key: "0a9fa1b3a7abb0ad9e017b400dc6749f"
server_key_iv: "2aa50cfb525141960a2e94215a6c53fe"
server_key_version: "encryptionkey@0"
I20260806 16:09:59.762262 30110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:09:59.766019 30128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:09:59.768165 30110 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 16:09:59.768428 30110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "0f5373aaab0349988cf7a85813c5b055"
format_stamp: "Formatted at 2026-08-06 16:09:59 on dist-test-slave-7z32"
server_key: "0a9fa1b3a7abb0ad9e017b400dc6749f"
server_key_iv: "2aa50cfb525141960a2e94215a6c53fe"
server_key_version: "encryptionkey@0"
I20260806 16:09:59.768620 30110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:09:59.789924 30110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:09:59.797574 30110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:09:59.797932 30110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:09:59.810617 30110 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:44761
I20260806 16:09:59.811268 30202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:44761 every 8 connection(s)
I20260806 16:09:59.812379 30110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:09:59.813726 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 30110
I20260806 16:09:59.813969 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:09:59.814405 8388 external_mini_cluster.cc:1428] Setting key 20b58b998d819a87b42b516a27ec5eb5
I20260806 16:09:59.816936 30203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:09:59.827431 30203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Bootstrap starting.
Aug 06 16:09:59 dist-test-slave-7z32 krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:09:59.832096 30203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Neither blocks nor log segments found. Creating new log.
I20260806 16:09:59.833514 30203 log.cc:824] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Log is configured to *not* fsync() on all Append() calls
I20260806 16:09:59.837493 30203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: No bootstrap required, opened a new log
I20260806 16:09:59.840819 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:09:59.816767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52038 (local address 127.8.49.62:44761)
0806 16:09:59.817449 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0806 16:09:59.817474 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:09:59.817533 (+ 59us) server_negotiation.cc:407] Connection header received
0806 16:09:59.818529 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:09:59.818555 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:09:59.819098 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:09:59.819564 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:09:59.821081 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:59.822980 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:59.825163 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:09:59.826089 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:09:59.826344 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:09:59.832604 (+ 6260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:09:59.832613 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:09:59.832643 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:09:59.832711 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:09:59.836178 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:59.837017 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:59.837024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:59.837032 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:59.837227 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:09:59.837650 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:09:59.837655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:09:59.837657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:09:59.838534 (+ 877us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:09:59.838898 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0806 16:09:59.839264 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 16:09:59.839710 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":225,"threads_started":1}
I20260806 16:09:59.843389 30203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:09:59.843741 30203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:09:59.843837 30203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5373aaab0349988cf7a85813c5b055, State: Initialized, Role: FOLLOWER
I20260806 16:09:59.844955 30203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:09:59.845209 30203 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:09:59.845283 30203 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:09:59.845427 30203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:09:59.846704 30203 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:09:59.847231 30203 leader_election.cc:302] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f5373aaab0349988cf7a85813c5b055; no voters:
I20260806 16:09:59.847726 30203 leader_election.cc:288] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:09:59.848076 30210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:09:59.848598 30210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 1 LEADER]: Becoming Leader. State: Replica: 0f5373aaab0349988cf7a85813c5b055, State: Running, Role: LEADER
I20260806 16:09:59.849177 30210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:09:59.849561 30203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:09:59.851711 30212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f5373aaab0349988cf7a85813c5b055. Latest consensus state: current_term: 1 leader_uuid: "0f5373aaab0349988cf7a85813c5b055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } } }
I20260806 16:09:59.851693 30211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f5373aaab0349988cf7a85813c5b055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } } }
I20260806 16:09:59.851927 30211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: This master's current role is: LEADER
I20260806 16:09:59.851927 30212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: This master's current role is: LEADER
W20260806 16:09:59.857300 30227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:09:59.857477 30227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:09:59.857563 30226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:09:59.858536 30226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:09:59.867241 30226 catalog_manager.cc:1347] Generated new cluster ID: 0ad42d4c1c684516a3204f7f51390739
I20260806 16:09:59.867389 30226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:09:59.895664 30226 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:09:59.897243 30226 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:09:59.907828 30226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Generated new TSK 0
I20260806 16:09:59.909003 30226 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:09:59.955754 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44761
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:10:00.130590 30238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:10:00.131091 30238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:10:00.131153 30238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:10:00.135854 30238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:10:00.135967 30238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:10:00.136039 30238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:10:00.142884 30238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:10:00.145252 30238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:10:00.147109 30238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:10:00.162294 30247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:00.162267 30243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:00.162405 30245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:00.162998 30238 server_base.cc:1034] running on GCE node
I20260806 16:10:00.163887 30238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:10:00.165122 30238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:10:00.168483 30238 hybrid_clock.cc:648] HybridClock initialized: now 1786032600168410 us; error 76 us; skew 500 ppm
Aug 06 16:10:00 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:10:00.172506 30238 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:10:00.174041 30238 webserver.cc:460] Webserver started at http://127.8.49.1:42193/ using document root <none> and password file <none>
I20260806 16:10:00.174975 30238 fs_manager.cc:359] Metadata directory not provided
I20260806 16:10:00.175098 30238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:10:00.175356 30238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:10:00.178225 30238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "dd2f763b45da40a1af4804d2908551e1"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "c92948ebb446997ecd4b3052915cbb8c"
server_key_iv: "b50914efd2f9f80f2c4b3387115343f1"
server_key_version: "encryptionkey@0"
I20260806 16:10:00.179175 30238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "dd2f763b45da40a1af4804d2908551e1"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "c92948ebb446997ecd4b3052915cbb8c"
server_key_iv: "b50914efd2f9f80f2c4b3387115343f1"
server_key_version: "encryptionkey@0"
I20260806 16:10:00.187017 30238 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260806 16:10:00.193001 30256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:00.195555 30238 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:10:00.195767 30238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "dd2f763b45da40a1af4804d2908551e1"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "c92948ebb446997ecd4b3052915cbb8c"
server_key_iv: "b50914efd2f9f80f2c4b3387115343f1"
server_key_version: "encryptionkey@0"
I20260806 16:10:00.195952 30238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:10:00.215989 30238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:10:00.221172 30238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:10:00.221412 30238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:10:00.222218 30238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:10:00.223497 30238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:10:00.223565 30238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:00.223654 30238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:10:00.223696 30238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:00.238284 30238 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:41157
I20260806 16:10:00.238344 30409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:41157 every 8 connection(s)
I20260806 16:10:00.239719 30238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:10:00.240929 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 30238
I20260806 16:10:00.241119 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:10:00.241914 8388 external_mini_cluster.cc:1428] Setting key e30362c19e6cb354e7611a78bb7691a6
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
Aug 06 16:10:00 dist-test-slave-7z32 krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:10:00.272563 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.245429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:53713 (local address 127.8.49.62:44761)
0806 16:10:00.245722 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:00.245729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:00.248819 (+ 3090us) server_negotiation.cc:407] Connection header received
0806 16:10:00.250299 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:00.250304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:00.250392 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:00.250514 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:00.253413 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.254350 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.256176 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.256449 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.256586 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.264628 (+ 8042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:00.264638 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:00.264645 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:00.264726 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:00.267414 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:00.268845 (+ 1431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:00.268850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:00.268852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:00.269052 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:00.269925 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:00.269935 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:00.269937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:00.270510 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:00.270649 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:00.272103 (+ 1454us) server_negotiation.cc:299] Negotiation successful
0806 16:10:00.272365 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 16:10:00.273773 30414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.246578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:53713)
0806 16:10:00.248543 (+ 1965us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:00.248619 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:00.249973 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:00.250805 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:00.250817 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:00.251687 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:10:00.252878 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.252911 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.254593 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.254597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.255976 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.255994 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.256687 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.256691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.256925 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.257926 (+ 1001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:10:00.257991 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:10:00.264185 (+ 6194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:10:00.267691 (+ 3506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:00.267703 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:00.267730 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:00.268556 (+ 826us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:00.269223 (+ 667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:00.269230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:00.269235 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:00.269531 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:00.271000 (+ 1469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:10:00.271012 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:10:00.271518 (+ 506us) client_negotiation.cc:769] Sending connection context
0806 16:10:00.271830 (+ 312us) client_negotiation.cc:240] Negotiation successful
0806 16:10:00.272348 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1446,"thread_start_us":371,"threads_started":1}
I20260806 16:10:00.276405 30410 heartbeater.cc:347] Connected to a master server at 127.8.49.62:44761
I20260806 16:10:00.276985 30410 heartbeater.cc:464] Registering TS with master...
I20260806 16:10:00.277879 30410 heartbeater.cc:511] Master 127.8.49.62:44761 requested a full tablet report, sending...
I20260806 16:10:00.281195 30150 ts_manager.cc:194] Registered new tserver with Master: dd2f763b45da40a1af4804d2908551e1 (127.8.49.1:41157)
I20260806 16:10:00.282801 30150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:53713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:10:00.292248 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44761
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:10:00.429616 30417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:10:00.429966 30417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:10:00.430042 30417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:10:00.434310 30417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:10:00.434433 30417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:10:00.434536 30417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:10:00.440339 30417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:10:00.441624 30417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:10:00.442765 30417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:10:00.454476 30424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:00.454497 30423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:00.454609 30427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:00.455096 30417 server_base.cc:1034] running on GCE node
I20260806 16:10:00.455664 30417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:10:00.456480 30417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:10:00.457738 30417 hybrid_clock.cc:648] HybridClock initialized: now 1786032600457731 us; error 68 us; skew 500 ppm
Aug 06 16:10:00 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:10:00.461621 30417 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:10:00.463358 30417 webserver.cc:460] Webserver started at http://127.8.49.2:37733/ using document root <none> and password file <none>
I20260806 16:10:00.464089 30417 fs_manager.cc:359] Metadata directory not provided
I20260806 16:10:00.464205 30417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:10:00.464483 30417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:10:00.467001 30417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "2a100999c1e54e76b3a624e6a0410ee2"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "591b39f46f1270548b41da134051e61e"
server_key_iv: "b0bbad8c74a552773e4afd152149fbb2"
server_key_version: "encryptionkey@0"
I20260806 16:10:00.467710 30417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "2a100999c1e54e76b3a624e6a0410ee2"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "591b39f46f1270548b41da134051e61e"
server_key_iv: "b0bbad8c74a552773e4afd152149fbb2"
server_key_version: "encryptionkey@0"
I20260806 16:10:00.472512 30417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:10:00.475829 30437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:00.477589 30417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:10:00.477783 30417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "2a100999c1e54e76b3a624e6a0410ee2"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "591b39f46f1270548b41da134051e61e"
server_key_iv: "b0bbad8c74a552773e4afd152149fbb2"
server_key_version: "encryptionkey@0"
I20260806 16:10:00.477907 30417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:10:00.501977 30417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:10:00.506570 30417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:10:00.506955 30417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:10:00.507822 30417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:10:00.509001 30417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:10:00.509083 30417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:00.509169 30417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:10:00.509227 30417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:00.522925 30417 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:37277
I20260806 16:10:00.522994 30588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:37277 every 8 connection(s)
I20260806 16:10:00.524222 30417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:10:00.524575 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 30417
I20260806 16:10:00.524744 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:10:00.525236 8388 external_mini_cluster.cc:1428] Setting key 733113de45385a7ea16bf0396a7bcc34
Aug 06 16:10:00 dist-test-slave-7z32 krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:10:00.546275 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.527171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:38475 (local address 127.8.49.62:44761)
0806 16:10:00.527354 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:00.527360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:00.529018 (+ 1658us) server_negotiation.cc:407] Connection header received
0806 16:10:00.530300 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:00.530305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:00.530383 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:00.530515 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:00.532624 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.533553 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.535786 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.536260 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.536442 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.540183 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:00.540191 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:00.540197 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:00.540394 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:00.542498 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:00.543554 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:00.543558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:00.543560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:00.543698 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:00.544393 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:00.544397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:00.544399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:00.544837 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:00.544979 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:00.545891 (+ 912us) server_negotiation.cc:299] Negotiation successful
0806 16:10:00.546096 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 16:10:00.547657 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.527676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:38475)
0806 16:10:00.528776 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:00.528845 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:00.529938 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:00.530793 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:00.530803 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:00.531424 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:10:00.532372 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.532395 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.533826 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.533832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.535515 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.535548 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.536536 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.536540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.536736 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.537644 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:10:00.537666 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:10:00.539814 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:10:00.542712 (+ 2898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:00.542720 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:00.542740 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:00.543320 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:00.543827 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:00.543830 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:00.543831 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:00.544145 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:00.545041 (+ 896us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:10:00.545048 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:10:00.545534 (+ 486us) client_negotiation.cc:769] Sending connection context
0806 16:10:00.545791 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 16:10:00.546284 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":201,"threads_started":1}
I20260806 16:10:00.549384 30589 heartbeater.cc:347] Connected to a master server at 127.8.49.62:44761
I20260806 16:10:00.549945 30589 heartbeater.cc:464] Registering TS with master...
I20260806 16:10:00.550806 30589 heartbeater.cc:511] Master 127.8.49.62:44761 requested a full tablet report, sending...
I20260806 16:10:00.552634 30150 ts_manager.cc:194] Registered new tserver with Master: 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277)
I20260806 16:10:00.553411 30150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:38475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:10:00.572044 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44761
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:10:00.726756 30596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:10:00.727217 30596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:10:00.727272 30596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:10:00.731871 30596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:10:00.731987 30596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:10:00.732059 30596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:10:00.738970 30596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:44761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:10:00.740694 30596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:10:00.742223 30596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:10:00.756508 30604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:00.756541 30607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:00.756502 30602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:00.756907 30596 server_base.cc:1034] running on GCE node
I20260806 16:10:00.757650 30596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:10:00.758476 30596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:10:00.759698 30596 hybrid_clock.cc:648] HybridClock initialized: now 1786032600759682 us; error 82 us; skew 500 ppm
Aug 06 16:10:00 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:10:00.763141 30596 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:10:00.764261 30596 webserver.cc:460] Webserver started at http://127.8.49.3:37711/ using document root <none> and password file <none>
I20260806 16:10:00.764950 30596 fs_manager.cc:359] Metadata directory not provided
I20260806 16:10:00.765050 30596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:10:00.765311 30596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:10:00.767755 30596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "c7a9f087e36047d89161ff4b2cc21926"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "27a89e7267a3a6abb60fbb1cc30a4b46"
server_key_iv: "65b70564e88645b2898ef774fb24ba5e"
server_key_version: "encryptionkey@0"
I20260806 16:10:00.768379 30596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "c7a9f087e36047d89161ff4b2cc21926"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "27a89e7267a3a6abb60fbb1cc30a4b46"
server_key_iv: "65b70564e88645b2898ef774fb24ba5e"
server_key_version: "encryptionkey@0"
I20260806 16:10:00.773281 30596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:10:00.776970 30614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:00.779167 30596 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:10:00.779389 30596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "c7a9f087e36047d89161ff4b2cc21926"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "27a89e7267a3a6abb60fbb1cc30a4b46"
server_key_iv: "65b70564e88645b2898ef774fb24ba5e"
server_key_version: "encryptionkey@0"
I20260806 16:10:00.779567 30596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:10:00.794354 30596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:10:00.798552 30596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:10:00.798867 30596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:10:00.799690 30596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:10:00.800942 30596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:10:00.801028 30596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:00.801119 30596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:10:00.801218 30596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:00.814343 30596 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:44471
I20260806 16:10:00.814414 30782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:44471 every 8 connection(s)
I20260806 16:10:00.815691 30596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:10:00.824908 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 30596
I20260806 16:10:00.825079 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:10:00.825438 8388 external_mini_cluster.cc:1428] Setting key 0d82b4584d898c819c259136e920616c
Aug 06 16:10:00 dist-test-slave-7z32 krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:10:00.837042 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.818241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:50769 (local address 127.8.49.62:44761)
0806 16:10:00.818482 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:00.818488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:00.820388 (+ 1900us) server_negotiation.cc:407] Connection header received
0806 16:10:00.821966 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:00.821971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:00.822061 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:00.822190 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:00.824462 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.825471 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.827614 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.827879 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.828021 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.831904 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:00.831909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:00.831912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:00.831956 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:00.833803 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:00.834792 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:00.834796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:00.834797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:00.835005 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:00.835548 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:00.835551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:00.835552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:00.835829 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:00.835922 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:00.836683 (+ 761us) server_negotiation.cc:299] Negotiation successful
0806 16:10:00.836852 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 16:10:00.838128 30785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.818705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:50769)
0806 16:10:00.820039 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:00.820127 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:00.821551 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:00.822520 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:00.822532 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:00.823251 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:10:00.824144 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.824180 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.825678 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.825682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.827395 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.827414 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.828093 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.828106 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.828324 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.829123 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:10:00.829146 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:10:00.831545 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:10:00.834056 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:00.834067 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:00.834100 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:00.834554 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:00.835151 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:00.835155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:00.835156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:00.835385 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:00.835974 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:10:00.835981 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:10:00.836327 (+ 346us) client_negotiation.cc:769] Sending connection context
0806 16:10:00.836528 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 16:10:00.836883 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"thread_start_us":220,"threads_started":1}
I20260806 16:10:00.840104 30783 heartbeater.cc:347] Connected to a master server at 127.8.49.62:44761
I20260806 16:10:00.840545 30783 heartbeater.cc:464] Registering TS with master...
I20260806 16:10:00.841414 30783 heartbeater.cc:511] Master 127.8.49.62:44761 requested a full tablet report, sending...
I20260806 16:10:00.843350 30151 ts_manager.cc:194] Registered new tserver with Master: c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471)
I20260806 16:10:00.844143 30151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:50769
I20260806 16:10:00.853817 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:10:00.867563 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.857294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.8.49.62:44761)
0806 16:10:00.857508 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:00.857516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:00.857754 (+ 238us) server_negotiation.cc:407] Connection header received
0806 16:10:00.857997 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:00.858002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:00.858084 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:00.858211 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:00.859354 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.860012 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.861122 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.861413 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.861570 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.863639 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:00.863644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:00.863647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:00.863684 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:00.865385 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:00.865921 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:00.865929 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:00.865931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:00.866016 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:00.866326 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:00.866330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:00.866332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:00.866689 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:00.866879 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:00.867169 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 16:10:00.867337 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 16:10:00.871908 30151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:10:00.876984 30151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:10:00.898926 30801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.892401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51966 (local address 127.8.49.2:37277)
0806 16:10:00.892765 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:00.892779 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:00.893573 (+ 794us) server_negotiation.cc:407] Connection header received
0806 16:10:00.893672 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:00.893679 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:00.893956 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:00.894159 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:10:00.894447 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.895560 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.897215 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.897874 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.898023 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.898215 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:00.898490 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 16:10:00.898676 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":154,"threads_started":1}
I20260806 16:10:00.898911 30798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.891082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41157 (local address 127.0.0.1:33108)
0806 16:10:00.892032 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:00.892106 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:00.892330 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:00.893281 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:00.893284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:00.893306 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:10:00.893434 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.893442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.894895 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.894898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.896759 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.896767 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.898398 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.898403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.898508 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.898538 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 16:10:00.898617 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 16:10:00.898724 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":185,"thread_start_us":115,"threads_started":1}
I20260806 16:10:00.899327 30800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.891368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51786 (local address 127.8.49.3:44471)
0806 16:10:00.891920 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:00.891936 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:00.892267 (+ 331us) server_negotiation.cc:407] Connection header received
0806 16:10:00.892649 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:00.892655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:00.892826 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:00.893016 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:10:00.893532 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.894979 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.896950 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.897946 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.898080 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.898290 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:00.898974 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 16:10:00.899153 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":182,"threads_started":1}
I20260806 16:10:00.899389 30799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.891229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33108 (local address 127.8.49.1:41157)
0806 16:10:00.892106 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:00.892115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:00.892252 (+ 137us) server_negotiation.cc:407] Connection header received
0806 16:10:00.892535 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:00.892543 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:00.892826 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:00.893016 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:10:00.893593 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.894733 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.896941 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.897946 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.898080 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.898290 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:00.899058 (+ 768us) server_negotiation.cc:299] Negotiation successful
0806 16:10:00.899221 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":192,"threads_started":1}
I20260806 16:10:00.898857 30796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.891205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:44471 (local address 127.0.0.1:51786)
0806 16:10:00.892145 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:00.892164 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:00.892390 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:00.893053 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:00.893060 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:00.893102 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:10:00.893311 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.893321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.895163 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.895167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.896746 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.896762 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.898320 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.898344 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.898457 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.898485 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 16:10:00.898565 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 16:10:00.898635 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":147,"thread_start_us":148,"threads_started":1}
I20260806 16:10:00.898866 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:00.892813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:37277 (local address 127.0.0.1:51966)
0806 16:10:00.893337 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:00.893353 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:00.893460 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:00.894157 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:00.894161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:00.894183 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:10:00.894293 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.894299 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.895718 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.895722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.897062 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:00.897075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:00.898037 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:00.898047 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:00.898186 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:00.898239 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 16:10:00.898407 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 16:10:00.898595 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":4,"thread_start_us":125,"threads_started":1}
I20260806 16:10:00.904199 30501 tablet_service.cc:1467] Processing CreateTablet for tablet e15abf5e83524d34a1ff3e1266c1cbf4 (DEFAULT_TABLE table=test-table [id=47725462f3494204aab78ea2e95c5958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:10:00.904488 30687 tablet_service.cc:1467] Processing CreateTablet for tablet e15abf5e83524d34a1ff3e1266c1cbf4 (DEFAULT_TABLE table=test-table [id=47725462f3494204aab78ea2e95c5958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:10:00.904488 30319 tablet_service.cc:1467] Processing CreateTablet for tablet e15abf5e83524d34a1ff3e1266c1cbf4 (DEFAULT_TABLE table=test-table [id=47725462f3494204aab78ea2e95c5958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:10:00.905685 30501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e15abf5e83524d34a1ff3e1266c1cbf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:10:00.906253 30687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e15abf5e83524d34a1ff3e1266c1cbf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:10:00.906250 30319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e15abf5e83524d34a1ff3e1266c1cbf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:10:00.914916 30803 tablet_bootstrap.cc:492] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: Bootstrap starting.
I20260806 16:10:00.917214 30806 tablet_bootstrap.cc:492] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: Bootstrap starting.
I20260806 16:10:00.918146 30804 tablet_bootstrap.cc:492] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Bootstrap starting.
I20260806 16:10:00.919286 30803 tablet_bootstrap.cc:654] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: Neither blocks nor log segments found. Creating new log.
I20260806 16:10:00.920229 30806 tablet_bootstrap.cc:654] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: Neither blocks nor log segments found. Creating new log.
I20260806 16:10:00.920459 30803 log.cc:824] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: Log is configured to *not* fsync() on all Append() calls
I20260806 16:10:00.921468 30804 tablet_bootstrap.cc:654] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Neither blocks nor log segments found. Creating new log.
I20260806 16:10:00.921912 30806 log.cc:824] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: Log is configured to *not* fsync() on all Append() calls
I20260806 16:10:00.922569 30804 log.cc:824] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Log is configured to *not* fsync() on all Append() calls
I20260806 16:10:00.923550 30803 tablet_bootstrap.cc:492] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: No bootstrap required, opened a new log
I20260806 16:10:00.923846 30803 ts_tablet_manager.cc:1397] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 16:10:00.924889 30806 tablet_bootstrap.cc:492] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: No bootstrap required, opened a new log
I20260806 16:10:00.925101 30804 tablet_bootstrap.cc:492] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: No bootstrap required, opened a new log
I20260806 16:10:00.925380 30804 ts_tablet_manager.cc:1397] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:10:00.925478 30806 ts_tablet_manager.cc:1397] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260806 16:10:00.928596 30803 raft_consensus.cc:348] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:00.928960 30803 raft_consensus.cc:374] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:10:00.929092 30803 raft_consensus.cc:729] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a100999c1e54e76b3a624e6a0410ee2, State: Initialized, Role: FOLLOWER
I20260806 16:10:00.929203 30806 raft_consensus.cc:348] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:00.929610 30806 raft_consensus.cc:374] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:10:00.929718 30806 raft_consensus.cc:729] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7a9f087e36047d89161ff4b2cc21926, State: Initialized, Role: FOLLOWER
I20260806 16:10:00.929687 30803 consensus_queue.cc:260] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:00.930339 30806 consensus_queue.cc:260] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:00.930286 30804 raft_consensus.cc:348] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:00.930671 30804 raft_consensus.cc:374] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:10:00.930738 30804 raft_consensus.cc:729] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2f763b45da40a1af4804d2908551e1, State: Initialized, Role: FOLLOWER
I20260806 16:10:00.930773 30589 heartbeater.cc:503] Master 127.8.49.62:44761 was elected leader, sending a full tablet report...
I20260806 16:10:00.931070 30803 ts_tablet_manager.cc:1428] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260806 16:10:00.931208 30804 consensus_queue.cc:260] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:00.932262 30806 ts_tablet_manager.cc:1428] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:10:00.932233 30410 heartbeater.cc:503] Master 127.8.49.62:44761 was elected leader, sending a full tablet report...
I20260806 16:10:00.932257 30804 ts_tablet_manager.cc:1428] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 16:10:00.932744 30783 heartbeater.cc:503] Master 127.8.49.62:44761 was elected leader, sending a full tablet report...
W20260806 16:10:00.993093 30411 tablet.cc:2369] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:10:01.025983 30590 tablet.cc:2369] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:10:01.067303 30784 tablet.cc:2369] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:10:01.083616 30814 raft_consensus.cc:482] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:10:01.083954 30814 raft_consensus.cc:504] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:01.085428 30814 leader_election.cc:288] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277), c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471)
I20260806 16:10:01.088444 30813 raft_consensus.cc:482] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:10:01.088685 30813 raft_consensus.cc:504] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:01.091320 30813 leader_election.cc:288] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd2f763b45da40a1af4804d2908551e1 (127.8.49.1:41157), c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471)
I20260806 16:10:01.092178 30801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:01.086081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:59693 (local address 127.8.49.2:37277)
0806 16:10:01.086446 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:01.086453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:01.087087 (+ 634us) server_negotiation.cc:407] Connection header received
0806 16:10:01.087304 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:01.087308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:01.087364 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:01.087467 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:10:01.087801 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.088667 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.090491 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.091160 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.091282 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:01.091371 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:01.091916 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 16:10:01.092022 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 16:10:01.092195 30800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:01.086102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:59183 (local address 127.8.49.3:44471)
0806 16:10:01.086250 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:01.086254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:01.086778 (+ 524us) server_negotiation.cc:407] Connection header received
0806 16:10:01.087038 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:01.087041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:01.087103 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:01.087191 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:10:01.087555 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.088143 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.090452 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.091250 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.091381 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:01.091474 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:01.091909 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 16:10:01.092027 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 16:10:01.093009 30817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:01.085972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:44471 (local address 127.8.49.1:59183)
0806 16:10:01.086626 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:01.086645 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:01.086872 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:01.087220 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:01.087223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:01.087245 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:10:01.087416 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:01.087424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.088428 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.088432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:01.090130 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:01.090144 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.091558 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.091562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:01.091717 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:01.091752 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 16:10:01.092668 (+ 916us) client_negotiation.cc:240] Negotiation successful
0806 16:10:01.092806 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":293,"thread_start_us":457,"threads_started":1}
I20260806 16:10:01.093231 30820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:01.085972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:37277 (local address 127.8.49.1:59693)
0806 16:10:01.086913 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:01.086928 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:01.087144 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:01.087480 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:01.087483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:01.087500 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:10:01.087672 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:01.087679 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.088818 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.088823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:01.090260 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:01.090280 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.091493 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.091497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:01.091607 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:01.091648 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 16:10:01.092811 (+ 1163us) client_negotiation.cc:240] Negotiation successful
0806 16:10:01.092947 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":45,"thread_start_us":336,"threads_started":1}
I20260806 16:10:01.094146 30526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15abf5e83524d34a1ff3e1266c1cbf4" candidate_uuid: "dd2f763b45da40a1af4804d2908551e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a100999c1e54e76b3a624e6a0410ee2" is_pre_election: true
I20260806 16:10:01.094333 30715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15abf5e83524d34a1ff3e1266c1cbf4" candidate_uuid: "dd2f763b45da40a1af4804d2908551e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7a9f087e36047d89161ff4b2cc21926" is_pre_election: true
I20260806 16:10:01.094434 30526 raft_consensus.cc:2454] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2f763b45da40a1af4804d2908551e1 in term 0.
I20260806 16:10:01.094683 30715 raft_consensus.cc:2454] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2f763b45da40a1af4804d2908551e1 in term 0.
I20260806 16:10:01.095109 30259 leader_election.cc:302] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a100999c1e54e76b3a624e6a0410ee2, dd2f763b45da40a1af4804d2908551e1; no voters:
I20260806 16:10:01.095649 30814 raft_consensus.cc:2781] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:10:01.095765 30814 raft_consensus.cc:482] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:10:01.095824 30814 raft_consensus.cc:3037] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:10:01.097455 30814 raft_consensus.cc:504] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:01.097744 30822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:01.091830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:44471 (local address 127.8.49.2:36257)
0806 16:10:01.092667 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:01.092683 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:01.092862 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:01.093267 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:01.093270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:01.093293 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:10:01.093470 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:01.093479 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.094638 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.094642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:01.096200 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:01.096251 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.097203 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.097216 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:01.097331 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:01.097355 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:10:01.097432 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 16:10:01.097532 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":225,"thread_start_us":92,"threads_started":1}
I20260806 16:10:01.097868 30799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:01.091988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:56491 (local address 127.8.49.1:41157)
0806 16:10:01.092119 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:01.092124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:01.092609 (+ 485us) server_negotiation.cc:407] Connection header received
0806 16:10:01.092840 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:01.092844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:01.092899 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:01.092949 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:10:01.093480 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.094264 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.096306 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.096773 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.096919 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:01.096996 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:01.097270 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 16:10:01.097379 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:10:01.097748 30821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:01.091524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41157 (local address 127.8.49.2:56491)
0806 16:10:01.092272 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:01.092504 (+ 232us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:01.092709 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:01.093115 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:01.093118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:01.093143 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:10:01.093300 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:01.093307 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.094407 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.094410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:01.096110 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:01.096130 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.096933 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.096937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:01.097074 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:01.097099 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 16:10:01.097478 (+ 379us) client_negotiation.cc:240] Negotiation successful
0806 16:10:01.097546 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":87,"thread_start_us":125,"threads_started":1}
I20260806 16:10:01.098157 30814 leader_election.cc:288] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 1 election: Requested vote from peers 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277), c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471)
I20260806 16:10:01.098946 30526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15abf5e83524d34a1ff3e1266c1cbf4" candidate_uuid: "dd2f763b45da40a1af4804d2908551e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a100999c1e54e76b3a624e6a0410ee2"
I20260806 16:10:01.099141 30526 raft_consensus.cc:3037] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:10:01.099105 30715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15abf5e83524d34a1ff3e1266c1cbf4" candidate_uuid: "dd2f763b45da40a1af4804d2908551e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7a9f087e36047d89161ff4b2cc21926"
I20260806 16:10:01.099159 30346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15abf5e83524d34a1ff3e1266c1cbf4" candidate_uuid: "2a100999c1e54e76b3a624e6a0410ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2f763b45da40a1af4804d2908551e1" is_pre_election: true
I20260806 16:10:01.099332 30715 raft_consensus.cc:3037] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:10:01.099481 30346 raft_consensus.cc:2379] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a100999c1e54e76b3a624e6a0410ee2 in current term 1: Already voted for candidate dd2f763b45da40a1af4804d2908551e1 in this term.
I20260806 16:10:01.100103 30800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:01.092230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:36257 (local address 127.8.49.3:44471)
0806 16:10:01.092628 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:01.092635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:01.092771 (+ 136us) server_negotiation.cc:407] Connection header received
0806 16:10:01.092992 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:01.092995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:01.093054 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:01.093115 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:10:01.093803 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.094505 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.096379 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.097044 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.097186 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:01.097302 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:01.097473 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 16:10:01.097666 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":142,"threads_started":1}
I20260806 16:10:01.100739 30526 raft_consensus.cc:2454] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2f763b45da40a1af4804d2908551e1 in term 1.
I20260806 16:10:01.100840 30714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e15abf5e83524d34a1ff3e1266c1cbf4" candidate_uuid: "2a100999c1e54e76b3a624e6a0410ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7a9f087e36047d89161ff4b2cc21926" is_pre_election: true
I20260806 16:10:01.101261 30259 leader_election.cc:302] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a100999c1e54e76b3a624e6a0410ee2, dd2f763b45da40a1af4804d2908551e1; no voters:
I20260806 16:10:01.101402 30715 raft_consensus.cc:2454] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2f763b45da40a1af4804d2908551e1 in term 1.
I20260806 16:10:01.101576 30814 raft_consensus.cc:2781] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:10:01.101795 30441 leader_election.cc:302] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a100999c1e54e76b3a624e6a0410ee2; no voters: c7a9f087e36047d89161ff4b2cc21926, dd2f763b45da40a1af4804d2908551e1
I20260806 16:10:01.101917 30814 raft_consensus.cc:686] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 1 LEADER]: Becoming Leader. State: Replica: dd2f763b45da40a1af4804d2908551e1, State: Running, Role: LEADER
I20260806 16:10:01.102372 30813 raft_consensus.cc:2726] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 16:10:01.102370 30814 consensus_queue.cc:237] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:01.107525 30150 catalog_manager.cc:5495] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd2f763b45da40a1af4804d2908551e1 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "dd2f763b45da40a1af4804d2908551e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } health_report { overall_health: UNKNOWN } } }
I20260806 16:10:01.149890 30799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:01.144583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.8.49.1:41157)
0806 16:10:01.144800 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:01.144805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:01.144858 (+ 53us) server_negotiation.cc:407] Connection header received
0806 16:10:01.144980 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:01.144983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:01.145064 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:01.145168 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:10:01.145762 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.146447 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.147726 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:01.147977 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:01.148125 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:01.148569 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:10:01.149276 (+ 707us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:10:01.149508 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:01.149616 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 16:10:01.149694 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 16:10:01.158624 30715 raft_consensus.cc:1261] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926 [term 1 FOLLOWER]: Refusing update from remote peer dd2f763b45da40a1af4804d2908551e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:10:01.158828 30526 raft_consensus.cc:1261] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 1 FOLLOWER]: Refusing update from remote peer dd2f763b45da40a1af4804d2908551e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:10:01.159546 30814 consensus_queue.cc:1035] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:10:01.159834 30825 consensus_queue.cc:1035] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:10:01.183960 30151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:10:01.184235 30151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:10:01.187638 30151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Sending DeleteTablet for 3 replicas of tablet e15abf5e83524d34a1ff3e1266c1cbf4
I20260806 16:10:01.188982 30501 tablet_service.cc:1514] Processing DeleteTablet for tablet e15abf5e83524d34a1ff3e1266c1cbf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:10:01 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:51966
I20260806 16:10:01.189287 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 30238
I20260806 16:10:01.189164 30687 tablet_service.cc:1514] Processing DeleteTablet for tablet e15abf5e83524d34a1ff3e1266c1cbf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:10:01 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:51786
I20260806 16:10:01.196133 30839 tablet_replica.cc:330] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: stopping tablet replica
I20260806 16:10:01.196552 30841 tablet_replica.cc:330] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: stopping tablet replica
I20260806 16:10:01.196834 30839 raft_consensus.cc:2229] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:10:01.197207 30841 raft_consensus.cc:2229] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:10:01.197434 30839 raft_consensus.cc:2258] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:10:01.197680 30841 raft_consensus.cc:2258] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:10:01.199822 30841 ts_tablet_manager.cc:1905] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:10:01.199831 30839 ts_tablet_manager.cc:1905] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:10:01.203426 30841 ts_tablet_manager.cc:1918] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:10:01.203928 30841 log.cc:1196] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/e15abf5e83524d34a1ff3e1266c1cbf4
I20260806 16:10:01.204463 30841 ts_tablet_manager.cc:1939] T e15abf5e83524d34a1ff3e1266c1cbf4 P c7a9f087e36047d89161ff4b2cc21926: Deleting consensus metadata
I20260806 16:10:01.204538 30839 ts_tablet_manager.cc:1918] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:10:01.204787 30839 log.cc:1196] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/e15abf5e83524d34a1ff3e1266c1cbf4
I20260806 16:10:01.205507 30839 ts_tablet_manager.cc:1939] T e15abf5e83524d34a1ff3e1266c1cbf4 P 2a100999c1e54e76b3a624e6a0410ee2: Deleting consensus metadata
I20260806 16:10:01.206548 30132 catalog_manager.cc:4841] TS c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471): tablet e15abf5e83524d34a1ff3e1266c1cbf4 (table test-table [id=47725462f3494204aab78ea2e95c5958]) successfully deleted
I20260806 16:10:01.207600 30131 catalog_manager.cc:4841] TS 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277): tablet e15abf5e83524d34a1ff3e1266c1cbf4 (table test-table [id=47725462f3494204aab78ea2e95c5958]) successfully deleted
W20260806 16:10:01.208374 30131 connection.cc:535] client connection to 127.8.49.1:41157 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 16:10:01.208632 30131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 16:10:01.208870 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 30417
I20260806 16:10:01.215754 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:01.210529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41157 (local address 127.0.0.1:33136)
0806 16:10:01.215332 (+ 4803us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:01.215538 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4567}
W20260806 16:10:01.216271 30131 catalog_manager.cc:4584] TS dd2f763b45da40a1af4804d2908551e1 (127.8.49.1:41157): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e15abf5e83524d34a1ff3e1266c1cbf4: Network error: Client connection negotiation failed: client connection to 127.8.49.1:41157: connect: Connection refused (error 111)
I20260806 16:10:01.233040 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 30596
I20260806 16:10:01.249529 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 30110
I20260806 16:10:21.272744 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:44761
--webserver_interface=127.8.49.62
--webserver_port=40337
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:44761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:10:21.449055 30843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:10:21.449508 30843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:10:21.449561 30843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:10:21.454628 30843 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 16:10:21.454746 30843 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 16:10:21.454764 30843 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 16:10:21.454780 30843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:10:21.454805 30843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:10:21.454906 30843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:10:21.454931 30843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:10:21.454949 30843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:10:21.464769 30843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:44761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:44761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:10:21.466519 30843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:10:21.468057 30843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:10:21.488974 30843 server_base.cc:1034] running on GCE node
W20260806 16:10:21.488888 30856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:21.488888 30855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:21.488968 30859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:21.490425 30843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:10:21.492558 30843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:10:21.493875 30843 hybrid_clock.cc:648] HybridClock initialized: now 1786032621493826 us; error 66 us; skew 500 ppm
Aug 06 16:10:21 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032621, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:10:21.506480 30843 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:10:21.508941 30843 webserver.cc:460] Webserver started at http://127.8.49.62:40337/ using document root <none> and password file <none>
I20260806 16:10:21.509840 30843 fs_manager.cc:359] Metadata directory not provided
I20260806 16:10:21.509960 30843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:10:21.516722 30843 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:10:21.521052 30871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:21.523662 30843 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260806 16:10:21.523985 30843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "0f5373aaab0349988cf7a85813c5b055"
format_stamp: "Formatted at 2026-08-06 16:09:59 on dist-test-slave-7z32"
server_key: "0a9fa1b3a7abb0ad9e017b400dc6749f"
server_key_iv: "2aa50cfb525141960a2e94215a6c53fe"
server_key_version: "encryptionkey@0"
I20260806 16:10:21.524973 30843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:10:21.567932 30843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:10:21.574249 30843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:10:21.574617 30843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:10:21.589007 30843 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:44761
I20260806 16:10:21.589466 30941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:44761 every 8 connection(s)
I20260806 16:10:21.592100 30843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:10:21.598724 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 30843
I20260806 16:10:21.599455 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:41157
--local_ip_for_outbound_sockets=127.8.49.1
--tserver_master_addrs=127.8.49.62:44761
--webserver_port=42193
--webserver_interface=127.8.49.1
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:10:21.601241 30942 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:10:21.603789 30942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Bootstrap starting.
I20260806 16:10:21.635398 30942 log.cc:824] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Log is configured to *not* fsync() on all Append() calls
I20260806 16:10:21.650291 30942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:10:21.650911 30942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Bootstrap complete.
I20260806 16:10:21.658972 30942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:10:21.659729 30942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5373aaab0349988cf7a85813c5b055, State: Initialized, Role: FOLLOWER
I20260806 16:10:21.660766 30942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:10:21.660987 30942 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:10:21.661083 30942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:10:21.661201 30942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:10:21.664446 30942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:10:21.664948 30942 leader_election.cc:302] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f5373aaab0349988cf7a85813c5b055; no voters:
I20260806 16:10:21.665436 30942 leader_election.cc:288] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 16:10:21.665841 30946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:10:21.666349 30946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 2 LEADER]: Becoming Leader. State: Replica: 0f5373aaab0349988cf7a85813c5b055, State: Running, Role: LEADER
I20260806 16:10:21.666945 30946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:10:21.667145 30942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:10:21.669777 30948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f5373aaab0349988cf7a85813c5b055. Latest consensus state: current_term: 2 leader_uuid: "0f5373aaab0349988cf7a85813c5b055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } } }
I20260806 16:10:21.670025 30948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: This master's current role is: LEADER
I20260806 16:10:21.670603 30947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0f5373aaab0349988cf7a85813c5b055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } } }
I20260806 16:10:21.670750 30947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: This master's current role is: LEADER
I20260806 16:10:21.673894 30962 catalog_manager.cc:1259] Loaded cluster ID: 0ad42d4c1c684516a3204f7f51390739
I20260806 16:10:21.674063 30962 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: loading cluster ID for follower catalog manager: success
I20260806 16:10:21.677071 30962 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: acquiring CA information for follower catalog manager: success
I20260806 16:10:21.679718 30962 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:10:21.680123 30963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:10:21.687601 30963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:10:21.688583 30963 catalog_manager.cc:1259] Loaded cluster ID: 0ad42d4c1c684516a3204f7f51390739
I20260806 16:10:21.688681 30963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:10:21.689841 30963 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:10:21.690502 30963 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Loaded TSK: 0
I20260806 16:10:21.713263 30963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Generated new TSK 1
I20260806 16:10:21.713788 30963 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:10:21.789562 30944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:10:21.789939 30944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:10:21.790185 30944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:10:21.795898 30944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:10:21.796008 30944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:10:21.796075 30944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:10:21.802613 30944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:41157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=42193
--tserver_master_addrs=127.8.49.62:44761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:10:21.804463 30944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:10:21.805985 30944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:10:21.823698 30975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:21.823805 30972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:21.824108 30944 server_base.cc:1034] running on GCE node
W20260806 16:10:21.824270 30973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:21.825001 30944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:10:21.825914 30944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:10:21.827214 30944 hybrid_clock.cc:648] HybridClock initialized: now 1786032621827176 us; error 59 us; skew 500 ppm
Aug 06 16:10:21 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032621, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:10:21.833077 30944 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:10:21.837558 30944 webserver.cc:460] Webserver started at http://127.8.49.1:42193/ using document root <none> and password file <none>
I20260806 16:10:21.839370 30944 fs_manager.cc:359] Metadata directory not provided
I20260806 16:10:21.839627 30944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:10:21.850912 30944 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.008s sys 0.004s
I20260806 16:10:21.856932 30983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:21.859748 30944 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.001s
I20260806 16:10:21.860375 30944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "dd2f763b45da40a1af4804d2908551e1"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "c92948ebb446997ecd4b3052915cbb8c"
server_key_iv: "b50914efd2f9f80f2c4b3387115343f1"
server_key_version: "encryptionkey@0"
I20260806 16:10:21.861655 30944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:10:21.894048 30944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:10:21.900485 30944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:10:21.900820 30944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:10:21.901902 30944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:10:21.904496 30993 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:10:21.910025 30944 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:10:21.910228 30944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 16:10:21.910353 30944 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:10:21.914467 30944 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:10:21.914618 30944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20260806 16:10:21.915220 30993 tablet_bootstrap.cc:492] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Bootstrap starting.
I20260806 16:10:21.933512 30944 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:41157
I20260806 16:10:21.933534 31126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:41157 every 8 connection(s)
I20260806 16:10:21.935518 30944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:10:21.935859 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 30944
I20260806 16:10:21.936514 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:37277
--local_ip_for_outbound_sockets=127.8.49.2
--tserver_master_addrs=127.8.49.62:44761
--webserver_port=37733
--webserver_interface=127.8.49.2
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:10:21.943768 30993 log.cc:824] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:10:21 dist-test-slave-7z32 krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032621, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:10:21.959383 30993 tablet_bootstrap.cc:492] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:10:21.960055 30993 tablet_bootstrap.cc:492] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Bootstrap complete.
I20260806 16:10:21.962579 30993 ts_tablet_manager.cc:1397] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Time spent bootstrapping tablet: real 0.048s user 0.028s sys 0.015s
I20260806 16:10:21.967749 30993 raft_consensus.cc:348] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:21.968885 30993 raft_consensus.cc:729] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2f763b45da40a1af4804d2908551e1, State: Initialized, Role: FOLLOWER
I20260806 16:10:21.969914 30993 consensus_queue.cc:260] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } } peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } }
I20260806 16:10:21.971199 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:21.940312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:42683 (local address 127.8.49.62:44761)
0806 16:10:21.942763 (+ 2451us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:21.942786 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:21.942888 (+ 102us) server_negotiation.cc:407] Connection header received
0806 16:10:21.944676 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:21.944693 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:21.945280 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:21.945771 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:21.948418 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:21.950081 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:21.952037 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:21.953165 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:21.953481 (+ 316us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:21.958648 (+ 5167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:21.958658 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:21.958680 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:21.958735 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:21.961554 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:21.964061 (+ 2507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:21.964079 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:21.964086 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:21.964375 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:21.964927 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:21.964930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:21.964932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:21.965494 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:21.965691 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:21.967530 (+ 1839us) server_negotiation.cc:299] Negotiation successful
0806 16:10:21.968559 (+ 1029us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2275,"thread_start_us":166,"threads_started":1}
I20260806 16:10:21.971199 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:21.940303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:42683)
0806 16:10:21.941935 (+ 1632us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:21.942009 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:21.943780 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:21.946464 (+ 2684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:21.946475 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:21.947207 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:10:21.948078 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:21.948109 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:21.950382 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:21.950387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:21.951740 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:21.951757 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:21.953477 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:21.953484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:21.954175 (+ 691us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:21.955328 (+ 1153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:10:21.955351 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:10:21.958196 (+ 2845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:10:21.962111 (+ 3915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:21.962121 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:21.962150 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:21.963179 (+ 1029us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:21.964529 (+ 1350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:21.964534 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:21.964536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:21.964781 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:21.965649 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:10:21.965659 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:10:21.966618 (+ 959us) client_negotiation.cc:769] Sending connection context
0806 16:10:21.967176 (+ 558us) client_negotiation.cc:240] Negotiation successful
0806 16:10:21.968559 (+ 1383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1349,"thread_start_us":211,"threads_started":1}
I20260806 16:10:21.972213 30993 ts_tablet_manager.cc:1428] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Time spent starting tablet: real 0.010s user 0.011s sys 0.001s
I20260806 16:10:21.975934 31128 heartbeater.cc:347] Connected to a master server at 127.8.49.62:44761
I20260806 16:10:21.976703 31128 heartbeater.cc:464] Registering TS with master...
I20260806 16:10:21.977769 31128 heartbeater.cc:511] Master 127.8.49.62:44761 requested a full tablet report, sending...
I20260806 16:10:21.985333 30895 ts_manager.cc:194] Registered new tserver with Master: dd2f763b45da40a1af4804d2908551e1 (127.8.49.1:41157)
I20260806 16:10:21.990965 30895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:42683
I20260806 16:10:21.995487 31128 heartbeater.cc:503] Master 127.8.49.62:44761 was elected leader, sending a full tablet report...
Aug 06 16:10:21 dist-test-slave-7z32 krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032621, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.1@KRBTEST.COM
I20260806 16:10:22.009167 31139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:21.990605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58188 (local address 127.8.49.1:41157)
0806 16:10:21.991354 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:21.991370 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:21.992101 (+ 731us) server_negotiation.cc:407] Connection header received
0806 16:10:21.992360 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:21.992368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:21.992658 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:21.992872 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:21.994023 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:21.995392 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:21.998672 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:21.999253 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:21.999419 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.003595 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:22.003609 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:22.003626 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:22.003657 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:22.006358 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:22.007247 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:22.007254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:22.007255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:22.007360 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:22.007738 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:22.007741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:22.007745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:22.008094 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:22.008279 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:22.008729 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 16:10:22.008941 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":264,"threads_started":1}
I20260806 16:10:22.009196 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:21.990373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41157 (local address 127.0.0.1:58188)
0806 16:10:21.991806 (+ 1433us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:21.991872 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:21.992122 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:21.992951 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:21.992959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:21.993595 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:10:21.993803 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:21.993815 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:21.995683 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:21.995689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:21.998463 (+ 2774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:21.998483 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:21.999403 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:21.999434 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:21.999607 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.000500 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:10:22.000519 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:10:22.003367 (+ 2848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:10:22.006578 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:22.006595 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:22.006602 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:22.007073 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:22.007506 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:22.007513 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:22.007519 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:22.007617 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:22.008247 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:10:22.008256 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:10:22.008428 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 16:10:22.008606 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 16:10:22.008925 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1131,"thread_start_us":173,"threads_started":1}
I20260806 16:10:22.011183 31050 tablet_service.cc:1514] Processing DeleteTablet for tablet e15abf5e83524d34a1ff3e1266c1cbf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:10:01 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:58188
I20260806 16:10:22.012581 31140 tablet_replica.cc:330] stopping tablet replica
I20260806 16:10:22.013146 31140 raft_consensus.cc:2229] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:10:22.013545 31140 raft_consensus.cc:2258] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:10:22.015409 31140 ts_tablet_manager.cc:1905] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:10:22.023164 31140 ts_tablet_manager.cc:1918] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:10:22.023460 31140 log.cc:1196] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal/wals/e15abf5e83524d34a1ff3e1266c1cbf4
I20260806 16:10:22.023862 31140 ts_tablet_manager.cc:1939] T e15abf5e83524d34a1ff3e1266c1cbf4 P dd2f763b45da40a1af4804d2908551e1: Deleting consensus metadata
I20260806 16:10:22.027918 30874 catalog_manager.cc:4841] TS dd2f763b45da40a1af4804d2908551e1 (127.8.49.1:41157): tablet e15abf5e83524d34a1ff3e1266c1cbf4 (table test-table [id=47725462f3494204aab78ea2e95c5958]) successfully deleted
W20260806 16:10:22.139727 31130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:10:22.140208 31130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:10:22.140270 31130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:10:22.145601 31130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:10:22.145727 31130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:10:22.145807 31130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:10:22.154675 31130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:37277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=37733
--tserver_master_addrs=127.8.49.62:44761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:10:22.156599 31130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:10:22.158138 31130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:10:22.171752 31152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:22.171825 31149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:22.171763 31150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:22.172098 31130 server_base.cc:1034] running on GCE node
I20260806 16:10:22.172937 31130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:10:22.174315 31130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:10:22.177714 31130 hybrid_clock.cc:648] HybridClock initialized: now 1786032622177656 us; error 54 us; skew 500 ppm
Aug 06 16:10:22 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:10:22.182053 31130 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:10:22.183444 31130 webserver.cc:460] Webserver started at http://127.8.49.2:37733/ using document root <none> and password file <none>
I20260806 16:10:22.184191 31130 fs_manager.cc:359] Metadata directory not provided
I20260806 16:10:22.184295 31130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:10:22.189910 31130 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:10:22.193347 31160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:22.195428 31130 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:10:22.195621 31130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "2a100999c1e54e76b3a624e6a0410ee2"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "591b39f46f1270548b41da134051e61e"
server_key_iv: "b0bbad8c74a552773e4afd152149fbb2"
server_key_version: "encryptionkey@0"
I20260806 16:10:22.196312 31130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:10:22.214965 31130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:10:22.221124 31130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:10:22.221390 31130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:10:22.222182 31130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:10:22.223312 31130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:10:22.223392 31130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:22.223470 31130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:10:22.223515 31130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:22.235168 31130 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:37277
I20260806 16:10:22.235232 31314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:37277 every 8 connection(s)
I20260806 16:10:22.236536 31130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:10:22.245174 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 31130
I20260806 16:10:22.245729 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:44471
--local_ip_for_outbound_sockets=127.8.49.3
--tserver_master_addrs=127.8.49.62:44761
--webserver_port=37711
--webserver_interface=127.8.49.3
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:10:22 dist-test-slave-7z32 krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:10:22.259833 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.239802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:42705 (local address 127.8.49.62:44761)
0806 16:10:22.240176 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:22.240181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:22.241681 (+ 1500us) server_negotiation.cc:407] Connection header received
0806 16:10:22.243049 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:22.243054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:22.243148 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:22.243278 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:22.244980 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.246070 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.248486 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.248990 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.249458 (+ 468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.253555 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:22.253564 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:22.253570 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:22.253640 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:22.255995 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:22.257161 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:22.257165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:22.257166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:22.257291 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:22.257927 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:22.257930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:22.257931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:22.258200 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:22.258322 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:22.259520 (+ 1198us) server_negotiation.cc:299] Negotiation successful
0806 16:10:22.259686 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260806 16:10:22.262123 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.240310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:42705)
0806 16:10:22.241430 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:22.241499 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:22.242653 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:22.243551 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:22.243561 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:22.244087 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:10:22.244777 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.244795 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.246346 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.246355 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.248179 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.248210 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.249507 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.249511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.249734 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.250632 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:10:22.250653 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:10:22.253127 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:10:22.256227 (+ 3100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:22.256244 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:22.256279 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:22.256952 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:22.257430 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:22.257436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:22.257440 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:22.257738 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:22.258371 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:10:22.258382 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:10:22.258987 (+ 605us) client_negotiation.cc:769] Sending connection context
0806 16:10:22.259454 (+ 467us) client_negotiation.cc:240] Negotiation successful
0806 16:10:22.260315 (+ 861us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":162,"threads_started":1}
I20260806 16:10:22.265206 31315 heartbeater.cc:347] Connected to a master server at 127.8.49.62:44761
I20260806 16:10:22.265607 31315 heartbeater.cc:464] Registering TS with master...
I20260806 16:10:22.266330 31315 heartbeater.cc:511] Master 127.8.49.62:44761 requested a full tablet report, sending...
I20260806 16:10:22.268253 30895 ts_manager.cc:194] Registered new tserver with Master: 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277)
I20260806 16:10:22.269342 30895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:42705
W20260806 16:10:22.432451 31318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:10:22.432886 31318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:10:22.432941 31318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:10:22.438073 31318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:10:22.438198 31318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:10:22.438275 31318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:10:22.445187 31318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:44471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=37711
--tserver_master_addrs=127.8.49.62:44761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:10:22.446959 31318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:10:22.448487 31318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:10:22.461930 31327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:22.461983 31330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:22.461946 31326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:22.462352 31318 server_base.cc:1034] running on GCE node
I20260806 16:10:22.463111 31318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:10:22.463961 31318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:10:22.465288 31318 hybrid_clock.cc:648] HybridClock initialized: now 1786032622465252 us; error 69 us; skew 500 ppm
Aug 06 16:10:22 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:10:22.470229 31318 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:10:22.471558 31318 webserver.cc:460] Webserver started at http://127.8.49.3:37711/ using document root <none> and password file <none>
I20260806 16:10:22.472388 31318 fs_manager.cc:359] Metadata directory not provided
I20260806 16:10:22.472534 31318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:10:22.477962 31318 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:10:22.481254 31340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:22.483058 31318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:10:22.483263 31318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "c7a9f087e36047d89161ff4b2cc21926"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "27a89e7267a3a6abb60fbb1cc30a4b46"
server_key_iv: "65b70564e88645b2898ef774fb24ba5e"
server_key_version: "encryptionkey@0"
I20260806 16:10:22.484011 31318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:10:22.495550 31318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:10:22.499559 31318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:10:22.499835 31318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:10:22.500558 31318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:10:22.501662 31318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:10:22.501739 31318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:22.501808 31318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:10:22.501860 31318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:22.514289 31318 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:44471
I20260806 16:10:22.514310 31489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:44471 every 8 connection(s)
I20260806 16:10:22.515918 31318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:10:22.520653 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 31318
Aug 06 16:10:22 dist-test-slave-7z32 krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:10:22.535486 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.521876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50848 (local address 127.8.49.62:44761)
0806 16:10:22.522182 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:22.522187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:22.522369 (+ 182us) server_negotiation.cc:407] Connection header received
0806 16:10:22.522498 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:22.522502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:22.522566 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:22.522712 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:22.523941 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.524773 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.526179 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.526663 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.526860 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.529625 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:22.529634 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:22.529638 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:22.529682 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:22.531955 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:22.532737 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:22.532742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:22.532744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:22.532873 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:22.533792 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:22.533796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:22.533807 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:22.534218 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:22.534546 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:22.534964 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 16:10:22.535192 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
I20260806 16:10:22.539146 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.519376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:45397 (local address 127.8.49.62:44761)
0806 16:10:22.519658 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:22.519663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:22.521065 (+ 1402us) server_negotiation.cc:407] Connection header received
0806 16:10:22.522932 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:22.522937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:22.523014 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:22.523126 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:22.524934 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.525814 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.527869 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.528397 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.528680 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.533094 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:22.533098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:22.533100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:22.533139 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:22.535665 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:22.536559 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:22.536563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:22.536565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:22.536686 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:22.537352 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:22.537357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:22.537359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:22.537683 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:22.537802 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:22.538702 (+ 900us) server_negotiation.cc:299] Negotiation successful
0806 16:10:22.538927 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 16:10:22.540014 31496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.519900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:45397)
0806 16:10:22.520758 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:22.520821 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:22.522585 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:22.523439 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:22.523447 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:22.523935 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:10:22.524649 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.524670 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.526084 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.526124 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.527615 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.527631 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.528912 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.528916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.529231 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.530267 (+ 1036us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:10:22.530296 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:10:22.532806 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:10:22.535851 (+ 3045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:22.535862 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:22.535892 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:22.536393 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:22.536944 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:22.536947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:22.536949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:22.537201 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:22.537836 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:10:22.537843 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:10:22.538244 (+ 401us) client_negotiation.cc:769] Sending connection context
0806 16:10:22.538538 (+ 294us) client_negotiation.cc:240] Negotiation successful
0806 16:10:22.538955 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":403,"threads_started":1}
I20260806 16:10:22.542223 31493 heartbeater.cc:347] Connected to a master server at 127.8.49.62:44761
I20260806 16:10:22.542574 31493 heartbeater.cc:464] Registering TS with master...
I20260806 16:10:22.543401 31493 heartbeater.cc:511] Master 127.8.49.62:44761 requested a full tablet report, sending...
I20260806 16:10:22.545028 30895 ts_manager.cc:194] Registered new tserver with Master: c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471)
I20260806 16:10:22.545827 30895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:45397
I20260806 16:10:22.551664 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:10:22.557634 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.552622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50850 (local address 127.8.49.62:44761)
0806 16:10:22.552817 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:22.552823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:22.553210 (+ 387us) server_negotiation.cc:407] Connection header received
0806 16:10:22.553349 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:22.553352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:22.553511 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:22.553650 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:10:22.553994 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.554721 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.556027 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.556290 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.556397 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.556889 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:10:22.557139 (+ 250us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 16:10:22.557477 (+ 338us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50850: token expired
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 16:10:22.557785 31132 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50850: token expired
W20260806 16:10:22.558223 30790 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.49.62:44761)
I20260806 16:10:22.567718 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.558773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50852 (local address 127.8.49.62:44761)
0806 16:10:22.558981 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:22.558984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:22.558997 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:10:22.559037 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:22.559039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:22.559091 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:22.559173 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:22.559926 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.560479 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.561370 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.561698 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.561814 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.563408 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:22.563412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:22.563414 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:22.563450 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:22.565488 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:22.566036 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:22.566039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:22.566041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:22.566158 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:22.566471 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:22.566474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:22.566475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:22.566750 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:22.566959 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:22.567366 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 16:10:22.567569 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260806 16:10:22.571738 30895 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:10:22.576083 30895 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:10:22.589988 31050 tablet_service.cc:1467] Processing CreateTablet for tablet 114b3571ccce4dc2bb2f073882af0b4d (DEFAULT_TABLE table=test-table [id=016e5557ba8148c297485b0f26cc0bd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:10:22.590965 31050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114b3571ccce4dc2bb2f073882af0b4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:10:22.593510 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.586382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:37277 (local address 127.0.0.1:46580)
0806 16:10:22.587001 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:22.587030 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:22.587226 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:22.588624 (+ 1398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:22.588627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:22.588643 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:10:22.588786 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.588796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.590233 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.590237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.591549 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.591560 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.592989 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.592993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.593098 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.593156 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 16:10:22.593229 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 16:10:22.593335 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":297,"threads_started":1}
I20260806 16:10:22.594053 31504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.586786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.8.49.2:37277)
0806 16:10:22.587582 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:22.587603 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:22.587644 (+ 41us) server_negotiation.cc:407] Connection header received
0806 16:10:22.587770 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:22.587781 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:22.588404 (+ 623us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:22.588627 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:10:22.588958 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.590080 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.591710 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.592577 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.592692 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.592914 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:22.593764 (+ 850us) server_negotiation.cc:299] Negotiation successful
0806 16:10:22.593897 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":318,"threads_started":1}
I20260806 16:10:22.598219 31510 tablet_bootstrap.cc:492] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: Bootstrap starting.
I20260806 16:10:22.598613 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.587845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.8.49.3:44471)
0806 16:10:22.590124 (+ 2279us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:22.590140 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:22.590171 (+ 31us) server_negotiation.cc:407] Connection header received
0806 16:10:22.590261 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:22.590267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:22.590542 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:22.590727 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:10:22.591236 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.593492 (+ 2256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.595334 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.597558 (+ 2224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.597764 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.597965 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:22.598294 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 16:10:22.598444 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2148,"thread_start_us":161,"threads_started":1}
I20260806 16:10:22.599406 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.587585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:44471 (local address 127.0.0.1:56228)
0806 16:10:22.588107 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:22.588128 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:22.588268 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:22.590991 (+ 2723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:22.590995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:22.591015 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:10:22.591102 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.591107 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.593641 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.593644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.595120 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.595139 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.597965 (+ 2826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.597969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.598096 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.598123 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:10:22.599141 (+ 1018us) client_negotiation.cc:240] Negotiation successful
0806 16:10:22.599254 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":181,"threads_started":1}
I20260806 16:10:22.599622 31510 tablet_bootstrap.cc:654] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: Neither blocks nor log segments found. Creating new log.
I20260806 16:10:22.599450 31221 tablet_service.cc:1467] Processing CreateTablet for tablet 114b3571ccce4dc2bb2f073882af0b4d (DEFAULT_TABLE table=test-table [id=016e5557ba8148c297485b0f26cc0bd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:10:22.601927 31221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114b3571ccce4dc2bb2f073882af0b4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:10:22.602334 31510 tablet_bootstrap.cc:492] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: No bootstrap required, opened a new log
I20260806 16:10:22.602455 31510 ts_tablet_manager.cc:1397] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 16:10:22.602995 31510 raft_consensus.cc:348] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:22.603142 31510 raft_consensus.cc:374] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:10:22.603194 31510 raft_consensus.cc:729] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2f763b45da40a1af4804d2908551e1, State: Initialized, Role: FOLLOWER
I20260806 16:10:22.603336 31510 consensus_queue.cc:260] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:22.603744 31510 ts_tablet_manager.cc:1428] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 16:10:22.603812 31408 tablet_service.cc:1467] Processing CreateTablet for tablet 114b3571ccce4dc2bb2f073882af0b4d (DEFAULT_TABLE table=test-table [id=016e5557ba8148c297485b0f26cc0bd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:10:22.606172 31408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114b3571ccce4dc2bb2f073882af0b4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:10:22.612972 31514 tablet_bootstrap.cc:492] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: Bootstrap starting.
I20260806 16:10:22.616988 31514 tablet_bootstrap.cc:654] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: Neither blocks nor log segments found. Creating new log.
I20260806 16:10:22.617866 31516 tablet_bootstrap.cc:492] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: Bootstrap starting.
I20260806 16:10:22.618217 31514 log.cc:824] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: Log is configured to *not* fsync() on all Append() calls
I20260806 16:10:22.620759 31516 tablet_bootstrap.cc:654] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: Neither blocks nor log segments found. Creating new log.
I20260806 16:10:22.621932 31514 tablet_bootstrap.cc:492] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: No bootstrap required, opened a new log
I20260806 16:10:22.622067 31516 log.cc:824] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: Log is configured to *not* fsync() on all Append() calls
I20260806 16:10:22.622274 31514 ts_tablet_manager.cc:1397] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260806 16:10:22.624890 31516 tablet_bootstrap.cc:492] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: No bootstrap required, opened a new log
I20260806 16:10:22.625188 31516 ts_tablet_manager.cc:1397] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 16:10:22.626573 31514 raft_consensus.cc:348] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:22.627067 31514 raft_consensus.cc:374] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:10:22.627163 31514 raft_consensus.cc:729] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a100999c1e54e76b3a624e6a0410ee2, State: Initialized, Role: FOLLOWER
I20260806 16:10:22.627794 31514 consensus_queue.cc:260] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:22.629050 31315 heartbeater.cc:503] Master 127.8.49.62:44761 was elected leader, sending a full tablet report...
I20260806 16:10:22.629360 31514 ts_tablet_manager.cc:1428] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: Time spent starting tablet: real 0.007s user 0.009s sys 0.000s
I20260806 16:10:22.629722 31516 raft_consensus.cc:348] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:22.630442 31516 raft_consensus.cc:374] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:10:22.630705 31516 raft_consensus.cc:729] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7a9f087e36047d89161ff4b2cc21926, State: Initialized, Role: FOLLOWER
I20260806 16:10:22.631383 31516 consensus_queue.cc:260] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:22.632510 31493 heartbeater.cc:503] Master 127.8.49.62:44761 was elected leader, sending a full tablet report...
I20260806 16:10:22.632941 31516 ts_tablet_manager.cc:1428] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
W20260806 16:10:22.688107 31129 tablet.cc:2369] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:10:22.694978 31520 raft_consensus.cc:482] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:10:22.695327 31520 raft_consensus.cc:504] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:22.696971 31520 leader_election.cc:288] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277), dd2f763b45da40a1af4804d2908551e1 (127.8.49.1:41157)
I20260806 16:10:22.702364 31504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.697775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:38293 (local address 127.8.49.2:37277)
0806 16:10:22.697969 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:22.697975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:22.697999 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:10:22.698090 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:22.698100 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:22.698163 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:22.698356 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:10:22.699123 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.699842 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.701232 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.701679 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.701815 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.701894 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:22.702129 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 16:10:22.702217 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 16:10:22.702364 31496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.697523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:37277 (local address 127.8.49.3:38293)
0806 16:10:22.697718 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:22.697739 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:22.697868 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:22.698403 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:22.698410 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:22.698442 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:10:22.698630 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.698643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.700034 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.700038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.701068 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.701080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.701824 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.701828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.701984 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.702005 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 16:10:22.702108 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 16:10:22.702206 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":1}
I20260806 16:10:22.703204 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.697932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41157 (local address 127.8.49.3:39561)
0806 16:10:22.698484 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:22.698505 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:22.698620 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:22.699123 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:22.699127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:22.699157 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:10:22.699296 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.699303 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.700510 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.700515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.701705 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:22.701723 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.702732 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.702736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:22.702870 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.702899 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 16:10:22.702979 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 16:10:22.703067 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":39,"thread_start_us":192,"threads_started":1}
I20260806 16:10:22.703256 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.698200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:39561 (local address 127.8.49.1:41157)
0806 16:10:22.698557 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:22.698574 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:22.698597 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:10:22.698768 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:22.698775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:22.698887 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:22.699009 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:10:22.699513 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.700256 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.701916 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.702519 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.702681 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.702954 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:22.703008 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 16:10:22.703096 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":129,"threads_started":1}
I20260806 16:10:22.703306 31253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114b3571ccce4dc2bb2f073882af0b4d" candidate_uuid: "c7a9f087e36047d89161ff4b2cc21926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a100999c1e54e76b3a624e6a0410ee2" is_pre_election: true
I20260806 16:10:22.703759 31253 raft_consensus.cc:2454] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7a9f087e36047d89161ff4b2cc21926 in term 0.
I20260806 16:10:22.704034 31071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114b3571ccce4dc2bb2f073882af0b4d" candidate_uuid: "c7a9f087e36047d89161ff4b2cc21926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2f763b45da40a1af4804d2908551e1" is_pre_election: true
I20260806 16:10:22.704524 31071 raft_consensus.cc:2454] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7a9f087e36047d89161ff4b2cc21926 in term 0.
I20260806 16:10:22.704561 31343 leader_election.cc:302] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a100999c1e54e76b3a624e6a0410ee2, c7a9f087e36047d89161ff4b2cc21926; no voters:
I20260806 16:10:22.705027 31520 raft_consensus.cc:2781] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:10:22.705152 31520 raft_consensus.cc:482] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:10:22.705195 31520 raft_consensus.cc:3037] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:10:22.710914 31520 raft_consensus.cc:504] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:22.711706 31520 leader_election.cc:288] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [CANDIDATE]: Term 1 election: Requested vote from peers 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277), dd2f763b45da40a1af4804d2908551e1 (127.8.49.1:41157)
I20260806 16:10:22.712800 31253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114b3571ccce4dc2bb2f073882af0b4d" candidate_uuid: "c7a9f087e36047d89161ff4b2cc21926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a100999c1e54e76b3a624e6a0410ee2"
I20260806 16:10:22.712810 31071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114b3571ccce4dc2bb2f073882af0b4d" candidate_uuid: "c7a9f087e36047d89161ff4b2cc21926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd2f763b45da40a1af4804d2908551e1"
I20260806 16:10:22.713053 31253 raft_consensus.cc:3037] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:10:22.713070 31071 raft_consensus.cc:3037] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:10:22.714541 31071 raft_consensus.cc:2454] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7a9f087e36047d89161ff4b2cc21926 in term 1.
I20260806 16:10:22.715098 31253 raft_consensus.cc:2454] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7a9f087e36047d89161ff4b2cc21926 in term 1.
I20260806 16:10:22.715207 31343 leader_election.cc:302] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7a9f087e36047d89161ff4b2cc21926, dd2f763b45da40a1af4804d2908551e1; no voters:
I20260806 16:10:22.715819 31520 raft_consensus.cc:2781] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:10:22.716341 31520 raft_consensus.cc:686] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 1 LEADER]: Becoming Leader. State: Replica: c7a9f087e36047d89161ff4b2cc21926, State: Running, Role: LEADER
I20260806 16:10:22.716892 31520 consensus_queue.cc:237] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:22.721623 30892 catalog_manager.cc:5495] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3). New cstate: current_term: 1 leader_uuid: "c7a9f087e36047d89161ff4b2cc21926" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } health_report { overall_health: UNKNOWN } } }
W20260806 16:10:22.738459 31316 tablet.cc:2369] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:10:22.760329 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.756344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.8.49.3:44471)
0806 16:10:22.756611 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:22.756616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:22.756630 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:10:22.756727 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:22.756730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:22.756792 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:22.756926 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:10:22.757196 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.757720 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.758593 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:22.758873 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:22.758996 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:22.759353 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:10:22.759872 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:10:22.760005 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:22.760094 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 16:10:22.760160 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260806 16:10:22.767287 31494 tablet.cc:2369] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:10:22.771942 31253 raft_consensus.cc:1261] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2 [term 1 FOLLOWER]: Refusing update from remote peer c7a9f087e36047d89161ff4b2cc21926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:10:22.772236 31071 raft_consensus.cc:1261] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Refusing update from remote peer c7a9f087e36047d89161ff4b2cc21926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:10:22.773031 31525 consensus_queue.cc:1035] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:10:22.773402 31520 consensus_queue.cc:1035] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:10:22.783475 31529 mvcc.cc:205] Tried to move back new op lower bound from 7315589622859431936 to 7315589622652731392. Current Snapshot: MvccSnapshot[applied={T|T < 7315589622859431936}]
I20260806 16:10:22.784993 31530 mvcc.cc:205] Tried to move back new op lower bound from 7315589622859431936 to 7315589622652731392. Current Snapshot: MvccSnapshot[applied={T|T < 7315589622859431936}]
I20260806 16:10:22.797583 30895 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:10:22.797783 30895 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:10:22.801884 30895 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Sending DeleteTablet for 3 replicas of tablet 114b3571ccce4dc2bb2f073882af0b4d
I20260806 16:10:22.803200 31050 tablet_service.cc:1514] Processing DeleteTablet for tablet 114b3571ccce4dc2bb2f073882af0b4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:10:22 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:58188
I20260806 16:10:22.803191 31221 tablet_service.cc:1514] Processing DeleteTablet for tablet 114b3571ccce4dc2bb2f073882af0b4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:10:22 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:46580
I20260806 16:10:22.803581 31408 tablet_service.cc:1514] Processing DeleteTablet for tablet 114b3571ccce4dc2bb2f073882af0b4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:10:22 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:56228
I20260806 16:10:22.803998 31544 tablet_replica.cc:330] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: stopping tablet replica
I20260806 16:10:22.804396 31545 tablet_replica.cc:330] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: stopping tablet replica
I20260806 16:10:22.804498 31544 raft_consensus.cc:2229] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:10:22.804538 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 30944
I20260806 16:10:22.804939 31545 raft_consensus.cc:2229] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:10:22.810505 31545 raft_consensus.cc:2258] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:10:22.813151 31544 raft_consensus.cc:2258] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:10:22.815937 31545 ts_tablet_manager.cc:1905] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:10:22.817806 31544 ts_tablet_manager.cc:1905] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:10:22.821792 31545 ts_tablet_manager.cc:1918] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:10:22.825728 31544 ts_tablet_manager.cc:1918] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:10:22.826139 31545 log.cc:1196] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/114b3571ccce4dc2bb2f073882af0b4d
I20260806 16:10:22.826697 31545 ts_tablet_manager.cc:1939] T 114b3571ccce4dc2bb2f073882af0b4d P c7a9f087e36047d89161ff4b2cc21926: Deleting consensus metadata
I20260806 16:10:22.828320 31544 log.cc:1196] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/114b3571ccce4dc2bb2f073882af0b4d
I20260806 16:10:22.828846 31544 ts_tablet_manager.cc:1939] T 114b3571ccce4dc2bb2f073882af0b4d P 2a100999c1e54e76b3a624e6a0410ee2: Deleting consensus metadata
I20260806 16:10:22.828778 30875 catalog_manager.cc:4841] TS c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471): tablet 114b3571ccce4dc2bb2f073882af0b4d (table test-table [id=016e5557ba8148c297485b0f26cc0bd9]) successfully deleted
I20260806 16:10:22.830435 30874 catalog_manager.cc:4841] TS 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277): tablet 114b3571ccce4dc2bb2f073882af0b4d (table test-table [id=016e5557ba8148c297485b0f26cc0bd9]) successfully deleted
W20260806 16:10:22.849150 30874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:41157 (error 108)
I20260806 16:10:22.850191 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 31130
I20260806 16:10:22.857137 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:22.855172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41157 (local address 127.0.0.1:58200)
0806 16:10:22.856858 (+ 1686us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:22.856986 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:41157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1534}
W20260806 16:10:22.857460 30874 catalog_manager.cc:4584] TS dd2f763b45da40a1af4804d2908551e1 (127.8.49.1:41157): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 114b3571ccce4dc2bb2f073882af0b4d: Network error: Client connection negotiation failed: client connection to 127.8.49.1:41157: connect: Connection refused (error 111)
I20260806 16:10:22.865906 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 31318
I20260806 16:10:22.885076 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 30843
I20260806 16:10:42.906801 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:44761
--webserver_interface=127.8.49.62
--webserver_port=40337
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:44761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:10:43.094622 31547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:10:43.095135 31547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:10:43.095189 31547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:10:43.100399 31547 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 16:10:43.100512 31547 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 16:10:43.100530 31547 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 16:10:43.100545 31547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:10:43.100566 31547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:10:43.100586 31547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:10:43.100605 31547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:10:43.100688 31547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:10:43.109848 31547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:44761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:44761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=40337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:10:43.111644 31547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:10:43.113125 31547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:10:43.125192 31553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:43.125236 31556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:43.125249 31552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:43.125382 31547 server_base.cc:1034] running on GCE node
I20260806 16:10:43.126433 31547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:10:43.128212 31547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:10:43.129468 31547 hybrid_clock.cc:648] HybridClock initialized: now 1786032643129418 us; error 76 us; skew 500 ppm
Aug 06 16:10:43 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032643, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:10:43.134264 31547 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:10:43.135805 31547 webserver.cc:460] Webserver started at http://127.8.49.62:40337/ using document root <none> and password file <none>
I20260806 16:10:43.137190 31547 fs_manager.cc:359] Metadata directory not provided
I20260806 16:10:43.137351 31547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:10:43.143579 31547 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:10:43.147747 31565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:43.150352 31547 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:10:43.150974 31547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "0f5373aaab0349988cf7a85813c5b055"
format_stamp: "Formatted at 2026-08-06 16:09:59 on dist-test-slave-7z32"
server_key: "0a9fa1b3a7abb0ad9e017b400dc6749f"
server_key_iv: "2aa50cfb525141960a2e94215a6c53fe"
server_key_version: "encryptionkey@0"
I20260806 16:10:43.152367 31547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:10:43.177119 31547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:10:43.183833 31547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:10:43.184217 31547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:10:43.195722 31547 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:44761
I20260806 16:10:43.196177 31636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:44761 every 8 connection(s)
I20260806 16:10:43.197726 31547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:10:43.203317 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 31547
I20260806 16:10:43.203594 31637 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:10:43.203924 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:41157
--local_ip_for_outbound_sockets=127.8.49.1
--tserver_master_addrs=127.8.49.62:44761
--webserver_port=42193
--webserver_interface=127.8.49.1
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:10:43.206382 31637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Bootstrap starting.
I20260806 16:10:43.226647 31637 log.cc:824] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Log is configured to *not* fsync() on all Append() calls
I20260806 16:10:43.240962 31637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:10:43.241524 31637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Bootstrap complete.
I20260806 16:10:43.247066 31637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:10:43.247650 31637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5373aaab0349988cf7a85813c5b055, State: Initialized, Role: FOLLOWER
I20260806 16:10:43.248315 31637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:10:43.248473 31637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:10:43.248602 31637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:10:43.248750 31637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 2 FOLLOWER]: Advancing to term 3
I20260806 16:10:43.251587 31637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:10:43.252071 31637 leader_election.cc:302] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f5373aaab0349988cf7a85813c5b055; no voters:
I20260806 16:10:43.252537 31637 leader_election.cc:288] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 16:10:43.252641 31642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 16:10:43.252955 31642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [term 3 LEADER]: Becoming Leader. State: Replica: 0f5373aaab0349988cf7a85813c5b055, State: Running, Role: LEADER
I20260806 16:10:43.253391 31642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } }
I20260806 16:10:43.254388 31637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:10:43.255517 31644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0f5373aaab0349988cf7a85813c5b055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } } }
I20260806 16:10:43.255785 31644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: This master's current role is: LEADER
I20260806 16:10:43.256376 31643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f5373aaab0349988cf7a85813c5b055. Latest consensus state: current_term: 3 leader_uuid: "0f5373aaab0349988cf7a85813c5b055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5373aaab0349988cf7a85813c5b055" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 44761 } } }
I20260806 16:10:43.256534 31643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055 [sys.catalog]: This master's current role is: LEADER
I20260806 16:10:43.257798 31652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:10:43.263608 31652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:10:43.264312 31652 catalog_manager.cc:1259] Loaded cluster ID: 0ad42d4c1c684516a3204f7f51390739
I20260806 16:10:43.264403 31652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:10:43.266693 31652 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:10:43.267566 31652 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Loaded TSK: 0
I20260806 16:10:43.267673 31652 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0f5373aaab0349988cf7a85813c5b055: Loaded TSK: 1
F20260806 16:10:43.268433 31652 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff2c789d9ad google::LogMessage::Fail() at ??:0
@ 0x7ff2c78a088a google::LogMessage::SendToLog() at ??:0
@ 0x7ff2c789d4de google::LogMessage::Flush() at ??:0
@ 0x7ff2c789ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff2ca56118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff2ca559f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff2d220c7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff2d21e699c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff2d21e7178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7ff2d2228df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ff2d224782b std::function<>::operator()() at ??:0
@ 0x7ff2d21e6b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ff2d21e7f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff2d21e4579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ff2d2228640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff2c9aef3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff2c9aefc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ff2c9af13f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff2c9ae0512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff2cb1286db start_thread at ??:0
@ 0x7ff2c639b61f clone at ??:0
W20260806 16:10:43.409680 31640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:10:43.410030 31640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:10:43.410079 31640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:10:43.416599 31640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:10:43.416719 31640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:10:43.416791 31640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:10:43.423942 31640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:41157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=42193
--tserver_master_addrs=127.8.49.62:44761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:10:43.425374 31640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:10:43.426620 31640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:10:43.441897 31696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:43.441824 31699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:43.441824 31697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:43.443022 31640 server_base.cc:1034] running on GCE node
I20260806 16:10:43.443883 31640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:10:43.445294 31640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:10:43.446628 31640 hybrid_clock.cc:648] HybridClock initialized: now 1786032643446565 us; error 74 us; skew 500 ppm
Aug 06 16:10:43 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032643, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:10:43.453094 31640 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:10:43.454530 31640 webserver.cc:460] Webserver started at http://127.8.49.1:42193/ using document root <none> and password file <none>
I20260806 16:10:43.455538 31640 fs_manager.cc:359] Metadata directory not provided
I20260806 16:10:43.455660 31640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:10:43.461445 31640 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:10:43.465107 31707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:43.467787 31640 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 16:10:43.468058 31640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "dd2f763b45da40a1af4804d2908551e1"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "c92948ebb446997ecd4b3052915cbb8c"
server_key_iv: "b50914efd2f9f80f2c4b3387115343f1"
server_key_version: "encryptionkey@0"
I20260806 16:10:43.468986 31640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:10:43.495471 31640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:10:43.499693 31640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:10:43.499938 31640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:10:43.500629 31640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:10:43.502688 31716 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:10:43.509060 31640 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:10:43.509287 31640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260806 16:10:43.509408 31640 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:10:43.512480 31640 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:10:43.512612 31640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260806 16:10:43.513032 31716 tablet_bootstrap.cc:492] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: Bootstrap starting.
I20260806 16:10:43.529496 31640 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:41157
I20260806 16:10:43.529855 31857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:41157 every 8 connection(s)
I20260806 16:10:43.531364 31640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:10:43.538533 31716 log.cc:824] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: Log is configured to *not* fsync() on all Append() calls
I20260806 16:10:43.539182 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 31640
I20260806 16:10:43.539915 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:37277
--local_ip_for_outbound_sockets=127.8.49.2
--tserver_master_addrs=127.8.49.62:44761
--webserver_port=37733
--webserver_interface=127.8.49.2
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:10:43.550925 31716 tablet_bootstrap.cc:492] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:10:43.551497 31716 tablet_bootstrap.cc:492] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: Bootstrap complete.
I20260806 16:10:43.553032 31716 ts_tablet_manager.cc:1397] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: Time spent bootstrapping tablet: real 0.040s user 0.030s sys 0.007s
I20260806 16:10:43.556988 31716 raft_consensus.cc:348] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:43.557662 31716 raft_consensus.cc:729] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2f763b45da40a1af4804d2908551e1, State: Initialized, Role: FOLLOWER
I20260806 16:10:43.558389 31716 consensus_queue.cc:260] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:43.559924 31716 ts_tablet_manager.cc:1428] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
W20260806 16:10:43.621011 31863 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:10:43.534206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:51593)
0806 16:10:43.534915 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:43.534987 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:43.536079 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:43.619768 (+ 83689us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":193,"threads_started":1}
W20260806 16:10:43.621946 31858 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:44761 (0 consecutive failures): Network error: Failed to ping master at 127.8.49.62:44761: Client connection negotiation failed: client connection to 127.8.49.62:44761: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 16:10:43.623584 31863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:43.623195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:36019)
0806 16:10:43.623382 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:43.623484 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 16:10:43.624644 31863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:43.624186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:37781)
0806 16:10:43.624458 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:43.624543 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260806 16:10:43.625001 31858 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:10:43.696487 31865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:10:43.697149 31865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:10:43.697397 31865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:10:43.704123 31865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:10:43.704237 31865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:10:43.704316 31865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:10:43.712425 31865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:37277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=37733
--tserver_master_addrs=127.8.49.62:44761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:10:43.713919 31865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:10:43.715272 31865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:10:43.730161 31873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:43.730152 31876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:43.730850 31865 server_base.cc:1034] running on GCE node
W20260806 16:10:43.730161 31872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:43.732620 31865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:10:43.734344 31865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:10:43.737767 31865 hybrid_clock.cc:648] HybridClock initialized: now 1786032643737701 us; error 84 us; skew 500 ppm
Aug 06 16:10:43 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032643, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:10:43.743144 31865 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:10:43.744825 31865 webserver.cc:460] Webserver started at http://127.8.49.2:37733/ using document root <none> and password file <none>
I20260806 16:10:43.745762 31865 fs_manager.cc:359] Metadata directory not provided
I20260806 16:10:43.745882 31865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:10:43.753443 31865 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:10:43.758219 31886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:43.761699 31865 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260806 16:10:43.762033 31865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "2a100999c1e54e76b3a624e6a0410ee2"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "591b39f46f1270548b41da134051e61e"
server_key_iv: "b0bbad8c74a552773e4afd152149fbb2"
server_key_version: "encryptionkey@0"
I20260806 16:10:43.763263 31865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:10:43.784431 31865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:10:43.791525 31865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:10:43.792547 31865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:10:43.794165 31865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:10:43.796144 31865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:10:43.796255 31865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:43.796325 31865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:10:43.796344 31865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:43.813875 31865 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:37277
I20260806 16:10:43.813970 32029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:37277 every 8 connection(s)
I20260806 16:10:43.815946 31865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:10:43.822184 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:43.819247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:40223)
0806 16:10:43.820638 (+ 1391us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:43.820868 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1023,"thread_start_us":201,"threads_started":1}
W20260806 16:10:43.823774 32030 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:44761 (0 consecutive failures): Network error: Failed to ping master at 127.8.49.62:44761: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
I20260806 16:10:43.825393 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:43.824717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:57599)
0806 16:10:43.824999 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:43.825111 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260806 16:10:43.826284 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 31865
I20260806 16:10:43.826933 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:44471
--local_ip_for_outbound_sockets=127.8.49.3
--tserver_master_addrs=127.8.49.62:44761
--webserver_port=37711
--webserver_interface=127.8.49.3
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:10:43.827747 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:43.827065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:45273)
0806 16:10:43.827343 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:43.827610 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
W20260806 16:10:43.828275 32030 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:10:43.978996 32033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:10:43.979385 32033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:10:43.979465 32033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:10:43.984973 32033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:10:43.985222 32033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:10:43.985330 32033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:10:43.991935 32033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:44471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=37711
--tserver_master_addrs=127.8.49.62:44761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:10:43.993664 32033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:10:43.995186 32033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:10:44.010020 32043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:10:44.009940 32038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:44.010299 32033 server_base.cc:1034] running on GCE node
W20260806 16:10:44.010130 32039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:10:44.011636 32033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:10:44.012809 32033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:10:44.014081 32033 hybrid_clock.cc:648] HybridClock initialized: now 1786032644013980 us; error 108 us; skew 500 ppm
Aug 06 16:10:44 dist-test-slave-7z32 krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032644, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:10:44.018411 32033 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:10:44.020418 32033 webserver.cc:460] Webserver started at http://127.8.49.3:37711/ using document root <none> and password file <none>
I20260806 16:10:44.021718 32033 fs_manager.cc:359] Metadata directory not provided
I20260806 16:10:44.021852 32033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:10:44.029902 32033 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:10:44.035507 32051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:44.038416 32033 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260806 16:10:44.039031 32033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "c7a9f087e36047d89161ff4b2cc21926"
format_stamp: "Formatted at 2026-08-06 16:10:00 on dist-test-slave-7z32"
server_key: "27a89e7267a3a6abb60fbb1cc30a4b46"
server_key_iv: "65b70564e88645b2898ef774fb24ba5e"
server_key_version: "encryptionkey@0"
I20260806 16:10:44.040853 32033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:10:44.067091 32033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:10:44.074119 32033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:10:44.074426 32033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:10:44.075409 32033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:10:44.076676 32033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:10:44.076732 32033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:44.076824 32033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:10:44.076985 32033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:10:44.093884 32033 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:44471
I20260806 16:10:44.093906 32202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:44471 every 8 connection(s)
I20260806 16:10:44.095671 32033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:10:44.101213 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 32033
I20260806 16:10:44.103037 32205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:44.099693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:53569)
0806 16:10:44.101410 (+ 1717us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:44.101689 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1127,"thread_start_us":788,"threads_started":1}
W20260806 16:10:44.104082 32203 heartbeater.cc:644] Failed to heartbeat to 127.8.49.62:44761 (0 consecutive failures): Network error: Failed to ping master at 127.8.49.62:44761: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
I20260806 16:10:44.105070 32205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:44.104681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:44947)
0806 16:10:44.104893 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:44.104981 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 16:10:44.106161 32205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:44.105678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:41521)
0806 16:10:44.105962 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:44.106059 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260806 16:10:44.106541 32203 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 16:10:44.627017 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:44.626107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:40427)
0806 16:10:44.626593 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:44.626698 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":116,"threads_started":1}
I20260806 16:10:44.830109 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:44.829385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:44615)
0806 16:10:44.829849 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:44.829946 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":127,"threads_started":1}
I20260806 16:10:44.976775 32213 raft_consensus.cc:482] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:10:44.977130 32213 raft_consensus.cc:504] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:44.979497 32213 leader_election.cc:288] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277), c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471)
Aug 06 16:10:44 dist-test-slave-7z32 krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032643, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.3@KRBTEST.COM
Aug 06 16:10:44 dist-test-slave-7z32 krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032643, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.2@KRBTEST.COM
I20260806 16:10:45.005606 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:44.979764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:37277 (local address 127.8.49.1:35085)
0806 16:10:44.980141 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:44.980163 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:44.980324 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:44.983479 (+ 3155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:44.983487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:44.984119 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:10:44.984800 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:44.984830 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:44.989641 (+ 4811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:44.989650 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:44.990355 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:44.990369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:44.991845 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:44.991850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:44.992046 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:44.992962 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:10:44.992985 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:10:44.995341 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:10:45.001477 (+ 6136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:45.001488 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:45.001514 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:45.002222 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:45.002760 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:45.002765 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:45.002767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:45.003132 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:45.004097 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:10:45.004106 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:10:45.004513 (+ 407us) client_negotiation.cc:769] Sending connection context
0806 16:10:45.004943 (+ 430us) client_negotiation.cc:240] Negotiation successful
0806 16:10:45.005364 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260806 16:10:45.005656 32215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:44.979946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:35085 (local address 127.8.49.2:37277)
0806 16:10:44.980734 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:44.980768 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:44.980844 (+ 76us) server_negotiation.cc:407] Connection header received
0806 16:10:44.982336 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:44.982347 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:44.982915 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:44.983320 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:44.986723 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:44.989395 (+ 2672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:44.990534 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:44.991625 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:44.992460 (+ 835us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:44.995897 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:44.995906 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:44.995937 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:44.996151 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:45.001195 (+ 5044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:45.002420 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:45.002428 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:45.002433 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:45.002611 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:45.003359 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:45.003367 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:45.003369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:45.003933 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:45.004133 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:45.005112 (+ 979us) server_negotiation.cc:299] Negotiation successful
0806 16:10:45.005424 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":98,"threads_started":1}
I20260806 16:10:45.005841 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:44.979879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:44471 (local address 127.8.49.1:60557)
0806 16:10:44.980295 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:44.980311 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:10:44.980413 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:10:44.984673 (+ 4260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:10:44.984681 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:10:44.985082 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:10:44.985473 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:44.985493 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:44.988250 (+ 2757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:44.988259 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:44.989550 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:10:44.989571 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:10:44.990564 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:10:44.990568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:10:44.990806 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:44.991885 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:10:44.991960 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:10:44.994612 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:10:45.002186 (+ 7574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:45.002194 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:45.002200 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:45.002609 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:45.003341 (+ 732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:10:45.003344 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:10:45.003347 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:10:45.003485 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:10:45.004937 (+ 1452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:10:45.004962 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:10:45.005126 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 16:10:45.005288 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 16:10:45.005532 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":150,"threads_started":1}
I20260806 16:10:45.006400 32216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:44.980656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:60557 (local address 127.8.49.3:44471)
0806 16:10:44.981173 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 16:10:44.981192 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:10:44.981244 (+ 52us) server_negotiation.cc:407] Connection header received
0806 16:10:44.982571 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:10:44.982583 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:10:44.983260 (+ 677us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:10:44.983863 (+ 603us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:10:44.985836 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:44.987968 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:44.989749 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:10:44.990381 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:10:44.990626 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:10:44.995810 (+ 5184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:10:44.995821 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:10:44.995953 (+ 132us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:10:44.996027 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:10:45.001934 (+ 5907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:45.002843 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:45.002855 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:45.002868 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:45.003188 (+ 320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:10:45.003595 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:10:45.003601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:10:45.003603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:10:45.004762 (+ 1159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:10:45.004995 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 16:10:45.005497 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 16:10:45.005950 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":182,"threads_started":1}
W20260806 16:10:45.012794 31712 leader_election.cc:341] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471): Not found: Tablet not found: 114b3571ccce4dc2bb2f073882af0b4d
W20260806 16:10:45.013559 31711 leader_election.cc:341] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277): Not found: Tablet not found: 114b3571ccce4dc2bb2f073882af0b4d
I20260806 16:10:45.013659 31711 leader_election.cc:302] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2f763b45da40a1af4804d2908551e1; no voters: 2a100999c1e54e76b3a624e6a0410ee2, c7a9f087e36047d89161ff4b2cc21926
I20260806 16:10:45.014002 32213 raft_consensus.cc:2726] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:10:45.108671 32217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:45.107942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:60865)
0806 16:10:45.108384 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:45.108466 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":135,"threads_started":1}
I20260806 16:10:45.629525 32218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:45.628667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:43403)
0806 16:10:45.629249 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:45.629375 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":112,"threads_started":1}
I20260806 16:10:45.832315 32219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:45.831489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:45137)
0806 16:10:45.832051 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:45.832161 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":171,"threads_started":1}
I20260806 16:10:46.110934 32221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:46.110022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:34413)
0806 16:10:46.110541 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:46.110651 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":98,"threads_started":1}
I20260806 16:10:46.631866 32222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:46.631174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:34245)
0806 16:10:46.631607 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:46.631711 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":128,"threads_started":1}
I20260806 16:10:46.834566 32223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:46.833817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:58335)
0806 16:10:46.834286 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:46.834407 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":135,"threads_started":1}
I20260806 16:10:47.055275 32224 raft_consensus.cc:482] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:10:47.055567 32224 raft_consensus.cc:504] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:47.056229 32224 leader_election.cc:288] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277), c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471)
W20260806 16:10:47.057508 31712 leader_election.cc:341] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471): Not found: Tablet not found: 114b3571ccce4dc2bb2f073882af0b4d
W20260806 16:10:47.057652 31711 leader_election.cc:341] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277): Not found: Tablet not found: 114b3571ccce4dc2bb2f073882af0b4d
I20260806 16:10:47.057865 31711 leader_election.cc:302] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2f763b45da40a1af4804d2908551e1; no voters: 2a100999c1e54e76b3a624e6a0410ee2, c7a9f087e36047d89161ff4b2cc21926
I20260806 16:10:47.058120 32224 raft_consensus.cc:2726] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:10:47.112617 32225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:47.111982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:60101)
0806 16:10:47.112378 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:47.112451 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":185,"threads_started":1}
I20260806 16:10:47.633812 32228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:47.633214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:40159)
0806 16:10:47.633622 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:47.633698 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":110,"threads_started":1}
I20260806 16:10:47.836359 32232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:47.835823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:48641)
0806 16:10:47.836181 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:47.836249 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":122,"threads_started":1}
I20260806 16:10:48.114617 32234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:48.113736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:60029)
0806 16:10:48.114247 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:48.114401 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":121,"threads_started":1}
I20260806 16:10:48.636226 32235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:48.635448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:53491)
0806 16:10:48.635847 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:48.636066 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":96,"threads_started":1}
I20260806 16:10:48.838238 32236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:48.837723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:58105)
0806 16:10:48.838040 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:48.838116 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":97,"threads_started":1}
I20260806 16:10:49.116714 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:49.115969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:55627)
0806 16:10:49.116416 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:49.116523 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":97,"threads_started":1}
I20260806 16:10:49.638185 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:49.637652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:33185)
0806 16:10:49.638013 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:49.638078 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":126,"threads_started":1}
I20260806 16:10:49.840211 32239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:49.839384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:40777)
0806 16:10:49.839942 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:49.840043 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":162,"threads_started":1}
I20260806 16:10:49.969681 32240 raft_consensus.cc:482] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:10:49.969907 32240 raft_consensus.cc:504] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:49.970453 32240 leader_election.cc:288] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277), c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471)
W20260806 16:10:49.971827 31711 leader_election.cc:341] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277): Not found: Tablet not found: 114b3571ccce4dc2bb2f073882af0b4d
W20260806 16:10:49.972023 31712 leader_election.cc:341] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471): Not found: Tablet not found: 114b3571ccce4dc2bb2f073882af0b4d
I20260806 16:10:49.972116 31712 leader_election.cc:302] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2f763b45da40a1af4804d2908551e1; no voters: 2a100999c1e54e76b3a624e6a0410ee2, c7a9f087e36047d89161ff4b2cc21926
I20260806 16:10:49.972363 32240 raft_consensus.cc:2726] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:10:50.119138 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:50.118286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:47475)
0806 16:10:50.118775 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:50.118971 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":137,"threads_started":1}
I20260806 16:10:50.640079 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:50.639395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:49159)
0806 16:10:50.639834 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:50.639927 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":130,"threads_started":1}
I20260806 16:10:50.842396 32243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:50.841531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:49599)
0806 16:10:50.842114 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:50.842229 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":105,"threads_started":1}
I20260806 16:10:51.121430 32244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:51.120782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:57893)
0806 16:10:51.121236 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:51.121310 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":99,"threads_started":1}
I20260806 16:10:51.641898 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:51.641339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:57585)
0806 16:10:51.641726 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:51.641792 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":169,"threads_started":1}
I20260806 16:10:51.844522 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:51.843741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:46451)
0806 16:10:51.844296 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:51.844388 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":143,"threads_started":1}
I20260806 16:10:52.123592 32248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:52.122804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:38223)
0806 16:10:52.123305 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:52.123467 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":113,"threads_started":1}
I20260806 16:10:52.644258 32249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:52.643572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:48647)
0806 16:10:52.644040 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:52.644137 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":131,"threads_started":1}
I20260806 16:10:52.846503 32250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:52.845988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:45731)
0806 16:10:52.846322 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:52.846397 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":134,"threads_started":1}
I20260806 16:10:53.125711 32251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:53.125050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:42503)
0806 16:10:53.125496 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:53.125594 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":159,"threads_started":1}
I20260806 16:10:53.646670 32252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:53.645878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:49705)
0806 16:10:53.646395 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:53.646508 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":106,"threads_started":1}
I20260806 16:10:53.848618 32253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:53.847864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:60659)
0806 16:10:53.848358 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:53.848467 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":132,"threads_started":1}
I20260806 16:10:54.127410 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:54.126814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:40573)
0806 16:10:54.127246 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:54.127303 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":131,"threads_started":1}
I20260806 16:10:54.440853 32255 raft_consensus.cc:482] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:10:54.441085 32255 raft_consensus.cc:504] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a100999c1e54e76b3a624e6a0410ee2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 37277 } } peers { permanent_uuid: "c7a9f087e36047d89161ff4b2cc21926" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 44471 } } peers { permanent_uuid: "dd2f763b45da40a1af4804d2908551e1" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41157 } }
I20260806 16:10:54.441881 32255 leader_election.cc:288] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277), c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471)
W20260806 16:10:54.443046 31712 leader_election.cc:341] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c7a9f087e36047d89161ff4b2cc21926 (127.8.49.3:44471): Not found: Tablet not found: 114b3571ccce4dc2bb2f073882af0b4d
W20260806 16:10:54.443181 31711 leader_election.cc:341] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a100999c1e54e76b3a624e6a0410ee2 (127.8.49.2:37277): Not found: Tablet not found: 114b3571ccce4dc2bb2f073882af0b4d
I20260806 16:10:54.443356 31711 leader_election.cc:302] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd2f763b45da40a1af4804d2908551e1; no voters: 2a100999c1e54e76b3a624e6a0410ee2, c7a9f087e36047d89161ff4b2cc21926
I20260806 16:10:54.443625 32255 raft_consensus.cc:2726] T 114b3571ccce4dc2bb2f073882af0b4d P dd2f763b45da40a1af4804d2908551e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:10:54.648912 32256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:54.648052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:46971)
0806 16:10:54.648653 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:54.648756 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":127,"threads_started":1}
I20260806 16:10:54.850813 32257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:54.849913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:48091)
0806 16:10:54.850612 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:54.850699 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":372,"threads_started":1}
I20260806 16:10:55.129699 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:55.128871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:43143)
0806 16:10:55.129379 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:55.129493 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":157,"threads_started":1}
I20260806 16:10:55.651016 32259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:55.650177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:40703)
0806 16:10:55.650707 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:55.650802 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":125,"threads_started":1}
I20260806 16:10:55.852978 32260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:55.852293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:39257)
0806 16:10:55.852740 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:55.852828 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":147,"threads_started":1}
I20260806 16:10:56.131908 32263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:56.131174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:48385)
0806 16:10:56.131687 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:56.131786 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":142,"threads_started":1}
I20260806 16:10:56.652998 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:56.652406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:53769)
0806 16:10:56.652801 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:56.652874 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":104,"threads_started":1}
I20260806 16:10:56.855304 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:56.854340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:49849)
0806 16:10:56.855047 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:56.855149 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":148,"threads_started":1}
I20260806 16:10:57.134104 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:57.133369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:35629)
0806 16:10:57.133865 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:57.133967 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":155,"threads_started":1}
I20260806 16:10:57.655198 32267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:57.654388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:39671)
0806 16:10:57.654917 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:57.655034 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":119,"threads_started":1}
I20260806 16:10:57.857128 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:57.856566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:60557)
0806 16:10:57.856932 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:57.857011 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":100,"threads_started":1}
I20260806 16:10:58.136643 32269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:58.135935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:35001)
0806 16:10:58.136374 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:58.136523 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":150,"threads_started":1}
I20260806 16:10:58.657727 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:58.656815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.1:54473)
0806 16:10:58.657268 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:58.657359 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":143,"threads_started":1}
I20260806 16:10:58.859354 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:58.858621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.2:43791)
0806 16:10:58.859135 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:58.859230 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":136,"threads_started":1}
I20260806 16:10:59.109687 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 31640
I20260806 16:10:59.139099 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:10:59.138318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:44761 (local address 127.8.49.3:37731)
0806 16:10:59.138777 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 16:10:59.138947 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.62:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":119,"threads_started":1}
I20260806 16:10:59.146603 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 31865
I20260806 16:10:59.166473 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 32033
2026-08-06T16:10:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (61223 ms)
[----------] 30 tests from SecurityITest (294306 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:10:59 dist-test-slave-7z32 krb5kdc[32276](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:10:59 dist-test-slave-7z32 krb5kdc[32276](info): set up 2 sockets
Aug 06 16:10:59 dist-test-slave-7z32 krb5kdc[32276](info): commencing operation
krb5kdc: starting...
W20260806 16:11:00.634145 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.433s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:00 dist-test-slave-7z32 krb5kdc[32276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:11:00.833685 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:41689
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:41689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:01.001664 32293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:01.002139 32293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:01.002194 32293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:01.007683 32293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:01.007826 32293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:01.007851 32293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:01.007870 32293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:01.007887 32293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:01.014914 32293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:41689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:41689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:01.016980 32293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:01.018608 32293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:11:01.033437 32293 server_base.cc:1034] running on GCE node
W20260806 16:11:01.033552 32299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:01.033349 32298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:01.033442 32301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:01.035305 32293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:01.038117 32293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:01.039568 32293 hybrid_clock.cc:648] HybridClock initialized: now 1786032661039548 us; error 80 us; skew 500 ppm
Aug 06 16:11:01 dist-test-slave-7z32 krb5kdc[32276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:01.050004 32293 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:11:01.051625 32293 webserver.cc:460] Webserver started at http://127.8.49.62:37907/ using document root <none> and password file <none>
I20260806 16:11:01.052625 32293 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:01.052734 32293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:01.053000 32293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:01.056267 32293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "2e294d336cb34db5a7cce04929eb818a"
format_stamp: "Formatted at 2026-08-06 16:11:01 on dist-test-slave-7z32"
server_key: "12c5978bd062af95abe8dbbadf123707"
server_key_iv: "f655c4e2a832020bed0fa65cf4f559a1"
server_key_version: "encryptionkey@0"
I20260806 16:11:01.057118 32293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "2e294d336cb34db5a7cce04929eb818a"
format_stamp: "Formatted at 2026-08-06 16:11:01 on dist-test-slave-7z32"
server_key: "12c5978bd062af95abe8dbbadf123707"
server_key_iv: "f655c4e2a832020bed0fa65cf4f559a1"
server_key_version: "encryptionkey@0"
I20260806 16:11:01.063808 32293 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:11:01.068199 32310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:01.070325 32293 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:11:01.070585 32293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "2e294d336cb34db5a7cce04929eb818a"
format_stamp: "Formatted at 2026-08-06 16:11:01 on dist-test-slave-7z32"
server_key: "12c5978bd062af95abe8dbbadf123707"
server_key_iv: "f655c4e2a832020bed0fa65cf4f559a1"
server_key_version: "encryptionkey@0"
I20260806 16:11:01.070808 32293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:01.088441 32293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:01.093297 32293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:01.093626 32293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:01.104187 32293 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:41689
I20260806 16:11:01.104346 32388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:41689 every 8 connection(s)
I20260806 16:11:01.105563 32293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:01.108081 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 32293
I20260806 16:11:01.108266 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:01.108590 8388 external_mini_cluster.cc:1428] Setting key 38efbda1fa4885bf81c2f190f5381d2d
I20260806 16:11:01.111416 32389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 16:11:01 dist-test-slave-7z32 krb5kdc[32276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:01.121178 32389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a: Bootstrap starting.
I20260806 16:11:01.125984 32389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:01.127782 32389 log.cc:824] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:01.129348 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:01.110503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33058 (local address 127.8.49.62:41689)
0806 16:11:01.111200 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:01.111230 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:01.111290 (+ 60us) server_negotiation.cc:407] Connection header received
0806 16:11:01.112387 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:01.112419 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:01.113001 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:01.113504 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:01.114541 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:01.116266 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:01.117480 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:01.118306 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:01.118579 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:01.122007 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:01.122017 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:01.122042 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:01.122098 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:01.124842 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:01.125578 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:01.125586 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:01.125595 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:01.125816 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:01.126286 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:01.126293 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:01.126295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:01.127203 (+ 908us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:01.127528 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:01.127927 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 16:11:01.128397 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":269,"threads_started":1}
I20260806 16:11:01.130995 32389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a: No bootstrap required, opened a new log
I20260806 16:11:01.135798 32389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e294d336cb34db5a7cce04929eb818a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41689 } }
I20260806 16:11:01.136157 32389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:01.136200 32389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e294d336cb34db5a7cce04929eb818a, State: Initialized, Role: FOLLOWER
I20260806 16:11:01.136829 32389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e294d336cb34db5a7cce04929eb818a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41689 } }
I20260806 16:11:01.137029 32389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:01.137112 32389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:01.137255 32389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:01.138609 32389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e294d336cb34db5a7cce04929eb818a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41689 } }
I20260806 16:11:01.139156 32389 leader_election.cc:302] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e294d336cb34db5a7cce04929eb818a; no voters:
I20260806 16:11:01.139616 32389 leader_election.cc:288] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:01.139938 32397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:01.140393 32397 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [term 1 LEADER]: Becoming Leader. State: Replica: 2e294d336cb34db5a7cce04929eb818a, State: Running, Role: LEADER
I20260806 16:11:01.141077 32397 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e294d336cb34db5a7cce04929eb818a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41689 } }
I20260806 16:11:01.141599 32389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:01.144096 32399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e294d336cb34db5a7cce04929eb818a. Latest consensus state: current_term: 1 leader_uuid: "2e294d336cb34db5a7cce04929eb818a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e294d336cb34db5a7cce04929eb818a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41689 } } }
I20260806 16:11:01.144186 32398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e294d336cb34db5a7cce04929eb818a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e294d336cb34db5a7cce04929eb818a" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41689 } } }
I20260806 16:11:01.144359 32399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:01.144398 32398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:01.144843 32410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:01.149058 32410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:01.157148 32410 catalog_manager.cc:1347] Generated new cluster ID: 4ea9d9bc347b4f93808351167ab1e9c3
I20260806 16:11:01.157269 32410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:01.166582 32410 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:01.167690 32410 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:01.173486 32410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a: Generated new TSK 0
I20260806 16:11:01.174374 32410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:11:01.223543 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41689
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:01.374204 32420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:01.374527 32420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:01.374575 32420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:01.378990 32420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:01.379089 32420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:01.379153 32420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:11:01.385869 32420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:01.387217 32420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:01.388415 32420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:01.401535 32427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:01.401530 32426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:01.401541 32429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:01.402160 32420 server_base.cc:1034] running on GCE node
I20260806 16:11:01.402654 32420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:01.403532 32420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:01.404870 32420 hybrid_clock.cc:648] HybridClock initialized: now 1786032661404816 us; error 71 us; skew 500 ppm
Aug 06 16:11:01 dist-test-slave-7z32 krb5kdc[32276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:01.408447 32420 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:11:01.409662 32420 webserver.cc:460] Webserver started at http://127.8.49.1:33707/ using document root <none> and password file <none>
I20260806 16:11:01.410392 32420 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:01.410498 32420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:01.410758 32420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:01.413182 32420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "7bd42bc4980c46658c27d4b06cd019c3"
format_stamp: "Formatted at 2026-08-06 16:11:01 on dist-test-slave-7z32"
server_key: "ec06cc58d3e4fff1105049fc02c2b63b"
server_key_iv: "59a1ac9255ad8237db05e0032bbbb47b"
server_key_version: "encryptionkey@0"
I20260806 16:11:01.413842 32420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "7bd42bc4980c46658c27d4b06cd019c3"
format_stamp: "Formatted at 2026-08-06 16:11:01 on dist-test-slave-7z32"
server_key: "ec06cc58d3e4fff1105049fc02c2b63b"
server_key_iv: "59a1ac9255ad8237db05e0032bbbb47b"
server_key_version: "encryptionkey@0"
I20260806 16:11:01.418756 32420 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 16:11:01.422257 32439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:01.424499 32420 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:01.424726 32420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "7bd42bc4980c46658c27d4b06cd019c3"
format_stamp: "Formatted at 2026-08-06 16:11:01 on dist-test-slave-7z32"
server_key: "ec06cc58d3e4fff1105049fc02c2b63b"
server_key_iv: "59a1ac9255ad8237db05e0032bbbb47b"
server_key_version: "encryptionkey@0"
I20260806 16:11:01.424906 32420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:01.440997 32420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:01.445200 32420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:01.445551 32420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:01.446302 32420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:01.447412 32420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:01.447501 32420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:01.447594 32420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:01.447646 32420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:01.460080 32420 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:37321
I20260806 16:11:01.460124 32586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:37321 every 8 connection(s)
I20260806 16:11:01.461297 32420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:11:01.464485 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 32420
I20260806 16:11:01.464672 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:11:01.465050 8388 external_mini_cluster.cc:1428] Setting key c62ce672f9ced5db3a7a63d628e89c11
Aug 06 16:11:01 dist-test-slave-7z32 krb5kdc[32276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
I20260806 16:11:01.483180 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:01.464538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:43459 (local address 127.8.49.62:41689)
0806 16:11:01.464803 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:01.464808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:01.466367 (+ 1559us) server_negotiation.cc:407] Connection header received
0806 16:11:01.468307 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:01.468310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:01.468372 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:01.468589 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:01.470590 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:01.471266 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:01.472517 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:01.472762 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:01.472885 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:01.477262 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:01.477269 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:01.477274 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:01.477314 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:01.479178 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:01.480462 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:01.480466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:01.480467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:01.480569 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:01.481289 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:01.481292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:01.481293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:01.481600 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:01.481706 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:01.482729 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0806 16:11:01.482987 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260806 16:11:01.484591 32589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:01.465154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:41689 (local address 127.8.49.1:43459)
0806 16:11:01.466107 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:01.466201 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:01.467957 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:01.468857 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:01.468872 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:01.469502 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:01.470352 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:01.470384 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:01.471428 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:01.471433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:01.472373 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:01.472383 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:01.472950 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:01.472954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:01.473142 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:01.474028 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:01.474054 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:01.476815 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:01.479389 (+ 2574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:01.479404 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:01.479441 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:01.480265 (+ 824us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:01.480708 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:01.480714 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:01.480717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:01.480982 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:01.481777 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:01.481792 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:01.482335 (+ 543us) client_negotiation.cc:769] Sending connection context
0806 16:11:01.482611 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 16:11:01.483273 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":254,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:11:01.486935 32587 heartbeater.cc:347] Connected to a master server at 127.8.49.62:41689
I20260806 16:11:01.487329 32587 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:01.488090 32587 heartbeater.cc:511] Master 127.8.49.62:41689 requested a full tablet report, sending...
I20260806 16:11:01.490463 32335 ts_manager.cc:194] Registered new tserver with Master: 7bd42bc4980c46658c27d4b06cd019c3 (127.8.49.1:37321)
I20260806 16:11:01.491948 32335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:43459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:11:01.511137 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41689
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:01.682575 32593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:01.683192 32593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:01.683301 32593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:01.689761 32593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:01.690054 32593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:01.690208 32593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:11:01.698312 32593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:01.700911 32593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:01.702730 32593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:01.721585 32603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:01.721695 32593 server_base.cc:1034] running on GCE node
W20260806 16:11:01.721647 32606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:01.721544 32600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:01.722749 32593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:01.723913 32593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:01.725343 32593 hybrid_clock.cc:648] HybridClock initialized: now 1786032661725248 us; error 108 us; skew 500 ppm
Aug 06 16:11:01 dist-test-slave-7z32 krb5kdc[32276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:01.730161 32593 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:11:01.731916 32593 webserver.cc:460] Webserver started at http://127.8.49.2:38449/ using document root <none> and password file <none>
I20260806 16:11:01.732812 32593 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:01.732908 32593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:01.733160 32593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:01.736526 32593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "52a9eb0e62344e6ba0ddd84075f1db04"
format_stamp: "Formatted at 2026-08-06 16:11:01 on dist-test-slave-7z32"
server_key: "d80def94ba8fcea1f74aa6174b14a68e"
server_key_iv: "2115ad065176c59efac79acd58d26fb3"
server_key_version: "encryptionkey@0"
I20260806 16:11:01.737488 32593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "52a9eb0e62344e6ba0ddd84075f1db04"
format_stamp: "Formatted at 2026-08-06 16:11:01 on dist-test-slave-7z32"
server_key: "d80def94ba8fcea1f74aa6174b14a68e"
server_key_iv: "2115ad065176c59efac79acd58d26fb3"
server_key_version: "encryptionkey@0"
I20260806 16:11:01.744916 32593 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260806 16:11:01.749939 32613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:01.752552 32593 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 16:11:01.752804 32593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "52a9eb0e62344e6ba0ddd84075f1db04"
format_stamp: "Formatted at 2026-08-06 16:11:01 on dist-test-slave-7z32"
server_key: "d80def94ba8fcea1f74aa6174b14a68e"
server_key_iv: "2115ad065176c59efac79acd58d26fb3"
server_key_version: "encryptionkey@0"
I20260806 16:11:01.752988 32593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:01.778180 32593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:01.783354 32593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:01.783619 32593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:01.784528 32593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:01.785897 32593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:01.785964 32593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:01.786020 32593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:01.786038 32593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:01.801820 32593 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:36317
I20260806 16:11:01.801927 300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:36317 every 8 connection(s)
I20260806 16:11:01.803401 32593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:11:01.805478 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 32593
I20260806 16:11:01.805640 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:11:01.805992 8388 external_mini_cluster.cc:1428] Setting key f227c5be90a5e48bdd608c3d613e8ca4
Aug 06 16:11:01 dist-test-slave-7z32 krb5kdc[32276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:11:01.826674 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:01.806808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:51891 (local address 127.8.49.62:41689)
0806 16:11:01.807089 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:01.807094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:01.808839 (+ 1745us) server_negotiation.cc:407] Connection header received
0806 16:11:01.810255 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:01.810259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:01.810330 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:01.810418 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:01.812016 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:01.812774 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:01.815285 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:01.815724 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:01.815895 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:01.820372 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:01.820381 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:01.820386 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:01.820444 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:01.822516 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:01.823646 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:01.823651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:01.823652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:01.823789 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:01.824621 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:01.824625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:01.824626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:01.824972 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:01.825082 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:01.826220 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0806 16:11:01.826450 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260806 16:11:01.827848 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:01.807679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:41689 (local address 127.8.49.2:51891)
0806 16:11:01.808620 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:01.808688 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:01.809897 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:01.810658 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:01.810667 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:01.811221 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:01.811802 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:01.811818 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:01.812994 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:01.813103 (+ 109us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:01.815021 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:01.815052 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:01.816190 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:01.816199 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:01.816452 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:01.817645 (+ 1193us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:01.817671 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:01.819968 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:01.822811 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:01.822893 (+ 82us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:01.822936 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:01.823421 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:01.823982 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:01.823986 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:01.823988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:01.824412 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:01.825152 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:01.825161 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:01.825770 (+ 609us) client_negotiation.cc:769] Sending connection context
0806 16:11:01.826014 (+ 244us) client_negotiation.cc:240] Negotiation successful
0806 16:11:01.826495 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":271,"threads_started":1}
I20260806 16:11:01.830381 301 heartbeater.cc:347] Connected to a master server at 127.8.49.62:41689
I20260806 16:11:01.830964 301 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:01.831799 301 heartbeater.cc:511] Master 127.8.49.62:41689 requested a full tablet report, sending...
I20260806 16:11:01.833788 32335 ts_manager.cc:194] Registered new tserver with Master: 52a9eb0e62344e6ba0ddd84075f1db04 (127.8.49.2:36317)
I20260806 16:11:01.834789 32335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:51891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:11:01.858049 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41689
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:33347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:02.040297 306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:02.040797 306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:02.040912 306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:02.047502 306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:02.047659 306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:02.047780 306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:11:02.056640 306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:33347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:02.058415 306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:02.060449 306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:02.076108 312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:02.076251 306 server_base.cc:1034] running on GCE node
W20260806 16:11:02.076145 315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:02.076162 313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:02.077170 306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:02.078244 306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:02.079653 306 hybrid_clock.cc:648] HybridClock initialized: now 1786032662079548 us; error 126 us; skew 500 ppm
Aug 06 16:11:02 dist-test-slave-7z32 krb5kdc[32276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032662, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:02.083763 306 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:11:02.084941 306 webserver.cc:460] Webserver started at http://127.8.49.3:43201/ using document root <none> and password file <none>
I20260806 16:11:02.085695 306 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:02.085799 306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:02.086066 306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:02.088611 306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "bd8f2a0ca5f34fd2949207e1f46026f1"
format_stamp: "Formatted at 2026-08-06 16:11:02 on dist-test-slave-7z32"
server_key: "d36373515f8ef18acab8fe28951f76ec"
server_key_iv: "403d79832ce54d3d799d13f29e906bb4"
server_key_version: "encryptionkey@0"
I20260806 16:11:02.089279 306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "bd8f2a0ca5f34fd2949207e1f46026f1"
format_stamp: "Formatted at 2026-08-06 16:11:02 on dist-test-slave-7z32"
server_key: "d36373515f8ef18acab8fe28951f76ec"
server_key_iv: "403d79832ce54d3d799d13f29e906bb4"
server_key_version: "encryptionkey@0"
I20260806 16:11:02.094491 306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 16:11:02.098565 323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:02.100826 306 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:02.101060 306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "bd8f2a0ca5f34fd2949207e1f46026f1"
format_stamp: "Formatted at 2026-08-06 16:11:02 on dist-test-slave-7z32"
server_key: "d36373515f8ef18acab8fe28951f76ec"
server_key_iv: "403d79832ce54d3d799d13f29e906bb4"
server_key_version: "encryptionkey@0"
I20260806 16:11:02.101225 306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:02.121551 306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:02.126094 306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:02.126372 306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:02.127245 306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:02.128337 306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:02.128415 306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:02.128527 306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:02.128573 306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:02.143368 306 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:40783
I20260806 16:11:02.143427 469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:40783 every 8 connection(s)
I20260806 16:11:02.144608 306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:11:02.152516 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 306
I20260806 16:11:02.152726 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:11:02.153054 8388 external_mini_cluster.cc:1428] Setting key f949597b75a4dba0e092d402bf355cc6
Aug 06 16:11:02 dist-test-slave-7z32 krb5kdc[32276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032662, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:02.170351 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.147896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:60635 (local address 127.8.49.62:41689)
0806 16:11:02.148141 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:02.148145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:02.149524 (+ 1379us) server_negotiation.cc:407] Connection header received
0806 16:11:02.152299 (+ 2775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:02.152304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:02.152406 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:02.152650 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:02.155458 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.156739 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.159398 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.159792 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.159898 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.164276 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:02.164281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:02.164283 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:02.164331 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:02.166127 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:02.167343 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:02.167347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:02.167349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:02.167479 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:02.168276 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:02.168280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:02.168282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:02.168632 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:02.168751 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:02.169883 (+ 1132us) server_negotiation.cc:299] Negotiation successful
0806 16:11:02.170103 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 16:11:02.171321 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.148373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:41689 (local address 127.8.49.3:60635)
0806 16:11:02.149240 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:02.149320 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:02.151824 (+ 2504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:02.152945 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:02.152957 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:02.153670 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:02.155099 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.155132 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.156989 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.156994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.158764 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.158788 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.160261 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.160264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.160624 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.161603 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:02.161628 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:02.163853 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:02.166337 (+ 2484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:02.166346 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:02.166404 (+ 58us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:02.167044 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:02.167757 (+ 713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:02.167762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:02.167765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:02.168106 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:02.168851 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:02.168861 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:02.169366 (+ 505us) client_negotiation.cc:769] Sending connection context
0806 16:11:02.169651 (+ 285us) client_negotiation.cc:240] Negotiation successful
0806 16:11:02.170119 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":178,"threads_started":1}
I20260806 16:11:02.173408 470 heartbeater.cc:347] Connected to a master server at 127.8.49.62:41689
I20260806 16:11:02.173808 470 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:02.174974 470 heartbeater.cc:511] Master 127.8.49.62:41689 requested a full tablet report, sending...
I20260806 16:11:02.176819 32335 ts_manager.cc:194] Registered new tserver with Master: bd8f2a0ca5f34fd2949207e1f46026f1 (127.8.49.3:40783)
I20260806 16:11:02.177588 32335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:60635
I20260806 16:11:02.183287 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:11:02.197001 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.186097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33060 (local address 127.8.49.62:41689)
0806 16:11:02.186374 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:02.186381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:02.186658 (+ 277us) server_negotiation.cc:407] Connection header received
0806 16:11:02.186868 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:02.186872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:02.186940 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:02.187039 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:02.187980 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.188618 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.189730 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.190008 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.190304 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.192349 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:02.192353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:02.192355 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:02.192399 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:02.194247 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:02.194964 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:02.194968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:02.194969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:02.195157 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:02.195645 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:02.195648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:02.195650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:02.196054 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:02.196202 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:02.196535 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 16:11:02.196730 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260806 16:11:02.201514 32335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:11:02.205413 32335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:11:02.227113 486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.219407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:40783 (local address 127.0.0.1:56778)
0806 16:11:02.220468 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:02.220538 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:02.220789 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:02.221744 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:02.221751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:02.221782 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:02.221982 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.221991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.223574 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.223579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.224645 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.224658 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.226392 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.226396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.226533 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.226580 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 16:11:02.226704 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 16:11:02.226881 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":178,"threads_started":1}
I20260806 16:11:02.227809 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.219559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.8.49.3:40783)
0806 16:11:02.220155 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:02.220170 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:02.221010 (+ 840us) server_negotiation.cc:407] Connection header received
0806 16:11:02.221119 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:02.221127 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:02.221529 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:02.221733 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:02.222174 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.223408 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.224803 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.225856 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.226133 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.226318 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:02.227533 (+ 1215us) server_negotiation.cc:299] Negotiation successful
0806 16:11:02.227652 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":158,"threads_started":1}
I20260806 16:11:02.228503 489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.220083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:37321 (local address 127.0.0.1:35222)
0806 16:11:02.220600 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:02.220631 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:02.221171 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:02.222078 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:02.222084 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:02.222112 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:02.222243 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.222250 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.225517 (+ 3267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.225521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.226895 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.226905 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.228125 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.228129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.228224 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.228251 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:11:02.228304 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 16:11:02.228380 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":135,"threads_started":1}
I20260806 16:11:02.229301 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.220273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35222 (local address 127.8.49.1:37321)
0806 16:11:02.221347 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:02.221360 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:02.221399 (+ 39us) server_negotiation.cc:407] Connection header received
0806 16:11:02.221515 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:02.221522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:02.221809 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:02.222994 (+ 1185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:02.223227 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.224902 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.227058 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.227953 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.228834 (+ 881us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.228998 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:02.229055 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 16:11:02.229156 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":895,"thread_start_us":95,"threads_started":1}
I20260806 16:11:02.230479 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.221069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41846 (local address 127.8.49.2:36317)
0806 16:11:02.221614 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:02.221629 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:02.222106 (+ 477us) server_negotiation.cc:407] Connection header received
0806 16:11:02.222213 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:02.222219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:02.222415 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:02.222981 (+ 566us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:02.223613 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.225554 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.227327 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.229322 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.229491 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.229683 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:02.229989 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 16:11:02.230125 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":186,"threads_started":1}
I20260806 16:11:02.230538 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.220529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:36317 (local address 127.0.0.1:41846)
0806 16:11:02.221811 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:02.221831 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:02.221973 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:02.223241 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:02.223245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:02.223269 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:02.223421 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.223432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.225780 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.225785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.227142 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.227156 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.229649 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.229653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.229795 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.229821 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:11:02.229930 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 16:11:02.230042 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1200,"thread_start_us":108,"threads_started":1}
I20260806 16:11:02.231521 384 tablet_service.cc:1467] Processing CreateTablet for tablet 8424db893f784e6e8648df028f34b68d (DEFAULT_TABLE table=test-table [id=8fe8f9679b2d405394296eea73bbb582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:02.231980 32497 tablet_service.cc:1467] Processing CreateTablet for tablet 8424db893f784e6e8648df028f34b68d (DEFAULT_TABLE table=test-table [id=8fe8f9679b2d405394296eea73bbb582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:02.232904 384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8424db893f784e6e8648df028f34b68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:02.232909 32497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8424db893f784e6e8648df028f34b68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:02.235231 32680 tablet_service.cc:1467] Processing CreateTablet for tablet 8424db893f784e6e8648df028f34b68d (DEFAULT_TABLE table=test-table [id=8fe8f9679b2d405394296eea73bbb582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:02.237753 32680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8424db893f784e6e8648df028f34b68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:02.241967 493 tablet_bootstrap.cc:492] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3: Bootstrap starting.
I20260806 16:11:02.242774 494 tablet_bootstrap.cc:492] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1: Bootstrap starting.
I20260806 16:11:02.245990 493 tablet_bootstrap.cc:654] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:02.246033 494 tablet_bootstrap.cc:654] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:02.247254 493 log.cc:824] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:02.247803 494 log.cc:824] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:02.249291 497 tablet_bootstrap.cc:492] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: Bootstrap starting.
I20260806 16:11:02.250310 493 tablet_bootstrap.cc:492] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3: No bootstrap required, opened a new log
I20260806 16:11:02.250628 493 ts_tablet_manager.cc:1397] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 16:11:02.250732 494 tablet_bootstrap.cc:492] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1: No bootstrap required, opened a new log
I20260806 16:11:02.251014 494 ts_tablet_manager.cc:1397] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.006s
I20260806 16:11:02.253463 497 tablet_bootstrap.cc:654] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:02.255057 497 log.cc:824] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:02.255003 494 raft_consensus.cc:348] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40783 } } peers { permanent_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36317 } } peers { permanent_uuid: "7bd42bc4980c46658c27d4b06cd019c3" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 37321 } }
I20260806 16:11:02.255448 494 raft_consensus.cc:374] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:02.255513 494 raft_consensus.cc:729] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8f2a0ca5f34fd2949207e1f46026f1, State: Initialized, Role: FOLLOWER
I20260806 16:11:02.256140 494 consensus_queue.cc:260] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40783 } } peers { permanent_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36317 } } peers { permanent_uuid: "7bd42bc4980c46658c27d4b06cd019c3" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 37321 } }
I20260806 16:11:02.256403 493 raft_consensus.cc:348] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40783 } } peers { permanent_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36317 } } peers { permanent_uuid: "7bd42bc4980c46658c27d4b06cd019c3" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 37321 } }
I20260806 16:11:02.256758 493 raft_consensus.cc:374] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:02.256829 493 raft_consensus.cc:729] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd42bc4980c46658c27d4b06cd019c3, State: Initialized, Role: FOLLOWER
I20260806 16:11:02.257385 470 heartbeater.cc:503] Master 127.8.49.62:41689 was elected leader, sending a full tablet report...
I20260806 16:11:02.257354 493 consensus_queue.cc:260] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40783 } } peers { permanent_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36317 } } peers { permanent_uuid: "7bd42bc4980c46658c27d4b06cd019c3" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 37321 } }
I20260806 16:11:02.257550 494 ts_tablet_manager.cc:1428] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260806 16:11:02.258284 497 tablet_bootstrap.cc:492] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: No bootstrap required, opened a new log
I20260806 16:11:02.258621 497 ts_tablet_manager.cc:1397] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260806 16:11:02.258665 493 ts_tablet_manager.cc:1428] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3: Time spent starting tablet: real 0.008s user 0.003s sys 0.005s
I20260806 16:11:02.258752 32587 heartbeater.cc:503] Master 127.8.49.62:41689 was elected leader, sending a full tablet report...
I20260806 16:11:02.263495 497 raft_consensus.cc:348] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40783 } } peers { permanent_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36317 } } peers { permanent_uuid: "7bd42bc4980c46658c27d4b06cd019c3" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 37321 } }
I20260806 16:11:02.263934 497 raft_consensus.cc:374] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:02.264007 497 raft_consensus.cc:729] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52a9eb0e62344e6ba0ddd84075f1db04, State: Initialized, Role: FOLLOWER
I20260806 16:11:02.264573 497 consensus_queue.cc:260] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40783 } } peers { permanent_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36317 } } peers { permanent_uuid: "7bd42bc4980c46658c27d4b06cd019c3" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 37321 } }
I20260806 16:11:02.265710 301 heartbeater.cc:503] Master 127.8.49.62:41689 was elected leader, sending a full tablet report...
I20260806 16:11:02.265928 497 ts_tablet_manager.cc:1428] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: Time spent starting tablet: real 0.007s user 0.008s sys 0.001s
I20260806 16:11:02.274396 500 raft_consensus.cc:482] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:11:02.274628 500 raft_consensus.cc:504] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40783 } } peers { permanent_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36317 } } peers { permanent_uuid: "7bd42bc4980c46658c27d4b06cd019c3" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 37321 } }
I20260806 16:11:02.276254 500 leader_election.cc:288] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52a9eb0e62344e6ba0ddd84075f1db04 (127.8.49.2:36317), 7bd42bc4980c46658c27d4b06cd019c3 (127.8.49.1:37321)
I20260806 16:11:02.282584 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.276846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:42161 (local address 127.8.49.2:36317)
0806 16:11:02.277128 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:02.277135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:02.277599 (+ 464us) server_negotiation.cc:407] Connection header received
0806 16:11:02.277693 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:02.277696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:02.277738 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:02.277815 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:02.278180 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.279019 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.280899 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.281680 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.281800 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.281880 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:02.282311 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 16:11:02.282401 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260806 16:11:02.282574 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.277056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:46629 (local address 127.8.49.1:37321)
0806 16:11:02.277205 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:02.277221 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:02.277246 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:11:02.277306 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:02.277309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:02.277356 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:02.277445 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:02.277929 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.278860 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.280899 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.281643 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.281795 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.281880 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:02.282311 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 16:11:02.282401 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:11:02.282618 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.276685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:37321 (local address 127.8.49.3:46629)
0806 16:11:02.276897 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:02.276922 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:02.277073 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:02.277503 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:02.277510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:02.277540 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:02.277757 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.277764 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.278982 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.278985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.280701 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.280717 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.281869 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.281874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.282066 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.282149 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 16:11:02.282267 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 16:11:02.282395 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":67,"spinlock_wait_cycles":9856}
I20260806 16:11:02.282627 506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.276771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:36317 (local address 127.8.49.3:42161)
0806 16:11:02.277441 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:02.277460 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:02.277586 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:02.277864 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:02.277870 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:02.277894 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:02.278021 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.278026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.279142 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.279145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.280716 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:02.280724 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.281869 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.281874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:02.282059 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.282150 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 16:11:02.282267 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 16:11:02.282377 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":130,"spinlock_wait_cycles":384,"thread_start_us":209,"threads_started":1}
I20260806 16:11:02.283762 32530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8424db893f784e6e8648df028f34b68d" candidate_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd42bc4980c46658c27d4b06cd019c3" is_pre_election: true
I20260806 16:11:02.283758 32708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8424db893f784e6e8648df028f34b68d" candidate_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" is_pre_election: true
I20260806 16:11:02.284219 32708 raft_consensus.cc:2454] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8f2a0ca5f34fd2949207e1f46026f1 in term 0.
I20260806 16:11:02.284219 32530 raft_consensus.cc:2454] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8f2a0ca5f34fd2949207e1f46026f1 in term 0.
I20260806 16:11:02.284958 329 leader_election.cc:302] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 52a9eb0e62344e6ba0ddd84075f1db04, 7bd42bc4980c46658c27d4b06cd019c3, bd8f2a0ca5f34fd2949207e1f46026f1; no voters:
I20260806 16:11:02.285494 500 raft_consensus.cc:2781] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:11:02.285666 500 raft_consensus.cc:482] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:11:02.285719 500 raft_consensus.cc:3037] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:02.287418 500 raft_consensus.cc:504] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40783 } } peers { permanent_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36317 } } peers { permanent_uuid: "7bd42bc4980c46658c27d4b06cd019c3" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 37321 } }
I20260806 16:11:02.288080 500 leader_election.cc:288] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [CANDIDATE]: Term 1 election: Requested vote from peers 52a9eb0e62344e6ba0ddd84075f1db04 (127.8.49.2:36317), 7bd42bc4980c46658c27d4b06cd019c3 (127.8.49.1:37321)
I20260806 16:11:02.289116 32708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8424db893f784e6e8648df028f34b68d" candidate_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52a9eb0e62344e6ba0ddd84075f1db04"
I20260806 16:11:02.289100 32530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8424db893f784e6e8648df028f34b68d" candidate_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd42bc4980c46658c27d4b06cd019c3"
I20260806 16:11:02.289423 32708 raft_consensus.cc:3037] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:02.289423 32530 raft_consensus.cc:3037] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:02.291208 32530 raft_consensus.cc:2454] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8f2a0ca5f34fd2949207e1f46026f1 in term 1.
I20260806 16:11:02.291208 32708 raft_consensus.cc:2454] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8f2a0ca5f34fd2949207e1f46026f1 in term 1.
I20260806 16:11:02.291747 329 leader_election.cc:302] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 52a9eb0e62344e6ba0ddd84075f1db04, 7bd42bc4980c46658c27d4b06cd019c3, bd8f2a0ca5f34fd2949207e1f46026f1; no voters:
I20260806 16:11:02.292059 500 raft_consensus.cc:2781] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:02.292430 500 raft_consensus.cc:686] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 1 LEADER]: Becoming Leader. State: Replica: bd8f2a0ca5f34fd2949207e1f46026f1, State: Running, Role: LEADER
I20260806 16:11:02.292902 500 consensus_queue.cc:237] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40783 } } peers { permanent_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36317 } } peers { permanent_uuid: "7bd42bc4980c46658c27d4b06cd019c3" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 37321 } }
I20260806 16:11:02.298224 32333 catalog_manager.cc:5495] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8f2a0ca5f34fd2949207e1f46026f1 (127.8.49.3). New cstate: current_term: 1 leader_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd8f2a0ca5f34fd2949207e1f46026f1" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 40783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bd42bc4980c46658c27d4b06cd019c3" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 37321 } health_report { overall_health: UNKNOWN } } }
W20260806 16:11:02.305716 302 tablet.cc:2369] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:11:02.334290 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.329193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56794 (local address 127.8.49.3:40783)
0806 16:11:02.329511 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:02.329515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:02.329537 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:11:02.329790 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:02.329848 (+ 58us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:02.329944 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:02.330142 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:02.330539 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.331256 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.332289 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.332548 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.332682 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.333103 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:02.333756 (+ 653us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:02.333934 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:02.333983 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 16:11:02.334066 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260806 16:11:02.343089 32708 raft_consensus.cc:1261] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04 [term 1 FOLLOWER]: Refusing update from remote peer bd8f2a0ca5f34fd2949207e1f46026f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:02.343139 32530 raft_consensus.cc:1261] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3 [term 1 FOLLOWER]: Refusing update from remote peer bd8f2a0ca5f34fd2949207e1f46026f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:02.344182 507 consensus_queue.cc:1035] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bd42bc4980c46658c27d4b06cd019c3" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:02.344570 500 consensus_queue.cc:1035] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52a9eb0e62344e6ba0ddd84075f1db04" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:02.368497 32333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:02.368888 32333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:02.372831 32333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e294d336cb34db5a7cce04929eb818a: Sending DeleteTablet for 3 replicas of tablet 8424db893f784e6e8648df028f34b68d
I20260806 16:11:02.374145 32497 tablet_service.cc:1514] Processing DeleteTablet for tablet 8424db893f784e6e8648df028f34b68d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:02 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:35222
I20260806 16:11:02.374173 384 tablet_service.cc:1514] Processing DeleteTablet for tablet 8424db893f784e6e8648df028f34b68d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:02 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:56778
I20260806 16:11:02.374173 32680 tablet_service.cc:1514] Processing DeleteTablet for tablet 8424db893f784e6e8648df028f34b68d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:02 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:41846
I20260806 16:11:02.374871 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 32420
I20260806 16:11:02.375057 526 tablet_replica.cc:330] T 8424db893f784e6e8648df028f34b68d P 7bd42bc4980c46658c27d4b06cd019c3: stopping tablet replica
I20260806 16:11:02.375695 524 tablet_replica.cc:330] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: stopping tablet replica
I20260806 16:11:02.376082 524 raft_consensus.cc:2229] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:11:02.376499 524 raft_consensus.cc:2258] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:02.378486 523 tablet_replica.cc:330] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1: stopping tablet replica
I20260806 16:11:02.378867 523 raft_consensus.cc:2229] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:11:02.379314 523 raft_consensus.cc:2258] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:02.381960 523 ts_tablet_manager.cc:1905] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:02.385273 524 ts_tablet_manager.cc:1905] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:02.385447 523 ts_tablet_manager.cc:1918] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:02.385610 523 log.cc:1196] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/8424db893f784e6e8648df028f34b68d
I20260806 16:11:02.386020 523 ts_tablet_manager.cc:1939] T 8424db893f784e6e8648df028f34b68d P bd8f2a0ca5f34fd2949207e1f46026f1: Deleting consensus metadata
I20260806 16:11:02.387683 32319 catalog_manager.cc:4841] TS bd8f2a0ca5f34fd2949207e1f46026f1 (127.8.49.3:40783): tablet 8424db893f784e6e8648df028f34b68d (table test-table [id=8fe8f9679b2d405394296eea73bbb582]) successfully deleted
I20260806 16:11:02.390786 524 ts_tablet_manager.cc:1918] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:02.391011 524 log.cc:1196] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/8424db893f784e6e8648df028f34b68d
I20260806 16:11:02.391430 524 ts_tablet_manager.cc:1939] T 8424db893f784e6e8648df028f34b68d P 52a9eb0e62344e6ba0ddd84075f1db04: Deleting consensus metadata
I20260806 16:11:02.393694 32319 catalog_manager.cc:4841] TS 52a9eb0e62344e6ba0ddd84075f1db04 (127.8.49.2:36317): tablet 8424db893f784e6e8648df028f34b68d (table test-table [id=8fe8f9679b2d405394296eea73bbb582]) successfully deleted
W20260806 16:11:02.397212 32314 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:37321 (error 108)
I20260806 16:11:02.397300 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 32593
I20260806 16:11:02.401383 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.400882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:37321 (local address 127.0.0.1:35230)
0806 16:11:02.401159 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:02.401244 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:37321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260806 16:11:02.401623 32314 catalog_manager.cc:4584] TS 7bd42bc4980c46658c27d4b06cd019c3 (127.8.49.1:37321): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8424db893f784e6e8648df028f34b68d: Network error: Client connection negotiation failed: client connection to 127.8.49.1:37321: connect: Connection refused (error 111)
I20260806 16:11:02.414814 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 306
I20260806 16:11:02.435717 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 32293
2026-08-06T16:11:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (3274 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T16:11:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:02Z Disabled control of system clock
I20260806 16:11:02.491308 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:45225
--webserver_interface=127.8.49.62
--webserver_port=0
--builtin_ntp_servers=127.8.49.20:41995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:45225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:11:02.630740 532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:02.631223 532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:02.631313 532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:02.636127 532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:02.636255 532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:02.636283 532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:02.636314 532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:02.636353 532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:02.641714 532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:45225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:45225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:02.643131 532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:02.644726 532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:02.655360 539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:02.655387 542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:02.655450 540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:02.655691 532 server_base.cc:1034] running on GCE node
I20260806 16:11:02.656504 532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:02.658107 532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:02.659409 532 hybrid_clock.cc:648] HybridClock initialized: now 1786032662659362 us; error 65 us; skew 500 ppm
I20260806 16:11:02.662940 532 webserver.cc:460] Webserver started at http://127.8.49.62:45587/ using document root <none> and password file <none>
I20260806 16:11:02.663866 532 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:02.663995 532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:02.664310 532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:02.667896 532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "bda7eeb10d0b4f9db9952fa559350403"
format_stamp: "Formatted at 2026-08-06 16:11:02 on dist-test-slave-7z32"
server_key: "8ac4b7a96cb8198e486c53269154a624"
server_key_iv: "62ed2747ec19f7a892ef96dd3ad90832"
server_key_version: "encryptionkey@0"
I20260806 16:11:02.669353 532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "bda7eeb10d0b4f9db9952fa559350403"
format_stamp: "Formatted at 2026-08-06 16:11:02 on dist-test-slave-7z32"
server_key: "8ac4b7a96cb8198e486c53269154a624"
server_key_iv: "62ed2747ec19f7a892ef96dd3ad90832"
server_key_version: "encryptionkey@0"
I20260806 16:11:02.676752 532 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260806 16:11:02.681205 548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:02.683780 532 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:11:02.684054 532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "bda7eeb10d0b4f9db9952fa559350403"
format_stamp: "Formatted at 2026-08-06 16:11:02 on dist-test-slave-7z32"
server_key: "8ac4b7a96cb8198e486c53269154a624"
server_key_iv: "62ed2747ec19f7a892ef96dd3ad90832"
server_key_version: "encryptionkey@0"
I20260806 16:11:02.684268 532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:02.715605 532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:02.716715 532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:02.717002 532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:02.727062 532 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:45225
I20260806 16:11:02.727136 616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:45225 every 8 connection(s)
I20260806 16:11:02.728504 532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:02.732786 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 532
I20260806 16:11:02.732956 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:02.733137 617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:02.733316 8388 external_mini_cluster.cc:1428] Setting key a0ee9d83469233a46246790cbb7e8c0e
I20260806 16:11:02.740474 617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403: Bootstrap starting.
I20260806 16:11:02.743714 617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:02.744654 617 log.cc:824] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:02.746441 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:02.735407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34018 (local address 127.8.49.62:45225)
0806 16:11:02.736375 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:02.736397 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:02.736447 (+ 50us) server_negotiation.cc:407] Connection header received
0806 16:11:02.737369 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:02.737392 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:02.737964 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:02.738390 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:02.739176 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.740306 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.741660 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:02.742295 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:02.742474 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:02.743667 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:02.743677 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:02.743703 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:02.744399 (+ 696us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:02.744488 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:02.744514 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:02.744625 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:02.744922 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 16:11:02.745358 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":270,"threads_started":1}
I20260806 16:11:02.747197 617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403: No bootstrap required, opened a new log
I20260806 16:11:02.751631 617 raft_consensus.cc:348] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda7eeb10d0b4f9db9952fa559350403" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45225 } }
I20260806 16:11:02.751937 617 raft_consensus.cc:374] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:02.752019 617 raft_consensus.cc:729] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda7eeb10d0b4f9db9952fa559350403, State: Initialized, Role: FOLLOWER
I20260806 16:11:02.752503 617 consensus_queue.cc:260] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda7eeb10d0b4f9db9952fa559350403" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45225 } }
I20260806 16:11:02.752674 617 raft_consensus.cc:388] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:02.752769 617 raft_consensus.cc:482] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:02.752900 617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:02.754016 617 raft_consensus.cc:504] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda7eeb10d0b4f9db9952fa559350403" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45225 } }
I20260806 16:11:02.754433 617 leader_election.cc:302] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bda7eeb10d0b4f9db9952fa559350403; no voters:
I20260806 16:11:02.754930 617 leader_election.cc:288] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:02.755309 623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:02.755699 623 raft_consensus.cc:686] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [term 1 LEADER]: Becoming Leader. State: Replica: bda7eeb10d0b4f9db9952fa559350403, State: Running, Role: LEADER
I20260806 16:11:02.756249 623 consensus_queue.cc:237] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda7eeb10d0b4f9db9952fa559350403" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45225 } }
I20260806 16:11:02.756556 617 sys_catalog.cc:564] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:02.759146 625 sys_catalog.cc:455] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bda7eeb10d0b4f9db9952fa559350403. Latest consensus state: current_term: 1 leader_uuid: "bda7eeb10d0b4f9db9952fa559350403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda7eeb10d0b4f9db9952fa559350403" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45225 } } }
I20260806 16:11:02.759388 625 sys_catalog.cc:458] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:02.759830 624 sys_catalog.cc:455] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bda7eeb10d0b4f9db9952fa559350403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bda7eeb10d0b4f9db9952fa559350403" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 45225 } } }
I20260806 16:11:02.760143 643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:02.760787 624 sys_catalog.cc:458] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:02.764040 643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:02.772044 643 catalog_manager.cc:1347] Generated new cluster ID: db254e3b6c104afb88dd8c42891abd28
I20260806 16:11:02.772166 643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:02.780472 643 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:02.782460 643 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:02.797891 643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403: Generated new TSK 0
I20260806 16:11:02.798941 643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:02.808939 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45225
--builtin_ntp_servers=127.8.49.20:41995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:11:02.988521 648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:02.988984 648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:02.989037 648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:02.994907 648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:02.995044 648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:02.995118 648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:11:03.002946 648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:03.004732 648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:03.007318 648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:03.022080 659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:03.022087 657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:03.022087 656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:03.022713 648 server_base.cc:1034] running on GCE node
I20260806 16:11:03.023382 648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:03.024386 648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:03.027906 648 hybrid_clock.cc:648] HybridClock initialized: now 1786032663027816 us; error 74 us; skew 500 ppm
I20260806 16:11:03.031553 648 webserver.cc:460] Webserver started at http://127.8.49.1:42485/ using document root <none> and password file <none>
I20260806 16:11:03.032497 648 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:03.032624 648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:03.032959 648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:03.036020 648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "e9d99ee7640f4089986fb20670069fea"
format_stamp: "Formatted at 2026-08-06 16:11:03 on dist-test-slave-7z32"
server_key: "351bff445e70d0a5772b435337678507"
server_key_iv: "f2a834fec36b2ecf817f84b92ebb2254"
server_key_version: "encryptionkey@0"
I20260806 16:11:03.036825 648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "e9d99ee7640f4089986fb20670069fea"
format_stamp: "Formatted at 2026-08-06 16:11:03 on dist-test-slave-7z32"
server_key: "351bff445e70d0a5772b435337678507"
server_key_iv: "f2a834fec36b2ecf817f84b92ebb2254"
server_key_version: "encryptionkey@0"
I20260806 16:11:03.042681 648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:11:03.047576 667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:03.049558 648 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 16:11:03.049816 648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "e9d99ee7640f4089986fb20670069fea"
format_stamp: "Formatted at 2026-08-06 16:11:03 on dist-test-slave-7z32"
server_key: "351bff445e70d0a5772b435337678507"
server_key_iv: "f2a834fec36b2ecf817f84b92ebb2254"
server_key_version: "encryptionkey@0"
I20260806 16:11:03.050014 648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:03.081957 648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:03.083294 648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:03.083583 648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:03.084631 648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:03.086201 648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:03.086303 648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:03.086395 648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:03.086442 648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:03.100014 648 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:38827
I20260806 16:11:03.100041 816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:38827 every 8 connection(s)
I20260806 16:11:03.101509 648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:11:03.104106 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 648
I20260806 16:11:03.104291 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:11:03.104650 8388 external_mini_cluster.cc:1428] Setting key 1f31d56e745afa8f5d0169791d4daf2d
I20260806 16:11:03.107573 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45225
--builtin_ntp_servers=127.8.49.20:41995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:11:03.114187 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.104285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:35823 (local address 127.8.49.62:45225)
0806 16:11:03.104486 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:03.104491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:03.106206 (+ 1715us) server_negotiation.cc:407] Connection header received
0806 16:11:03.107959 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:03.107963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:03.108025 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:03.108150 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:03.109245 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.109813 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.111071 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.111473 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.111606 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.113384 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:03.113391 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:03.113397 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:03.113419 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:03.113451 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:03.113465 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:03.113565 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:03.113850 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 16:11:03.113947 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 16:11:03.115607 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.104891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45225 (local address 127.8.49.1:35823)
0806 16:11:03.105969 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:03.106041 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:03.107631 (+ 1590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:03.108377 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:03.108387 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:03.108489 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:03.109073 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.109086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.109969 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.109974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.110926 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.110937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.111764 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.111770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.112033 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.112977 (+ 944us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:11:03.112979 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:03.113040 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:11:03.113153 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:03.113619 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:03.113630 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:03.113637 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:11:03.113820 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 16:11:03.114208 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":173,"threads_started":1}
I20260806 16:11:03.117654 818 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45225
I20260806 16:11:03.118095 818 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:03.119098 818 heartbeater.cc:511] Master 127.8.49.62:45225 requested a full tablet report, sending...
I20260806 16:11:03.121644 573 ts_manager.cc:194] Registered new tserver with Master: e9d99ee7640f4089986fb20670069fea (127.8.49.1:38827)
I20260806 16:11:03.123327 573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.1:35823
W20260806 16:11:03.259539 821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:03.259972 821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:03.260090 821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:03.264634 821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:03.264722 821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:03.264784 821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:11:03.271453 821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:03.273267 821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:03.275205 821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:03.288110 826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:03.288237 830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:03.288224 827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:03.288573 821 server_base.cc:1034] running on GCE node
I20260806 16:11:03.289216 821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:03.290050 821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:03.293399 821 hybrid_clock.cc:648] HybridClock initialized: now 1786032663293285 us; error 123 us; skew 500 ppm
I20260806 16:11:03.296176 821 webserver.cc:460] Webserver started at http://127.8.49.2:41073/ using document root <none> and password file <none>
I20260806 16:11:03.296989 821 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:03.297132 821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:03.297421 821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:03.300235 821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "a8995df30e0c4221a2cb26624d005e78"
format_stamp: "Formatted at 2026-08-06 16:11:03 on dist-test-slave-7z32"
server_key: "96a1c3c274187d3df800c90a06cc2712"
server_key_iv: "e2be81ed5e391b0e94fbed1e96a3cad5"
server_key_version: "encryptionkey@0"
I20260806 16:11:03.300931 821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "a8995df30e0c4221a2cb26624d005e78"
format_stamp: "Formatted at 2026-08-06 16:11:03 on dist-test-slave-7z32"
server_key: "96a1c3c274187d3df800c90a06cc2712"
server_key_iv: "e2be81ed5e391b0e94fbed1e96a3cad5"
server_key_version: "encryptionkey@0"
I20260806 16:11:03.306005 821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:11:03.309908 836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:03.311939 821 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:11:03.312175 821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "a8995df30e0c4221a2cb26624d005e78"
format_stamp: "Formatted at 2026-08-06 16:11:03 on dist-test-slave-7z32"
server_key: "96a1c3c274187d3df800c90a06cc2712"
server_key_iv: "e2be81ed5e391b0e94fbed1e96a3cad5"
server_key_version: "encryptionkey@0"
I20260806 16:11:03.312340 821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:03.347154 821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:03.348078 821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:03.348341 821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:03.349085 821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:03.350276 821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:03.350363 821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:03.350448 821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:03.350495 821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:03.362746 821 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:36761
I20260806 16:11:03.362912 983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:36761 every 8 connection(s)
I20260806 16:11:03.363955 821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:11:03.372323 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 821
I20260806 16:11:03.372529 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:11:03.372886 8388 external_mini_cluster.cc:1428] Setting key bc8be9e85e325717d22ae3202ce60d38
I20260806 16:11:03.375880 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45225
--builtin_ntp_servers=127.8.49.20:41995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:11:03.377862 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.366742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:47403 (local address 127.8.49.62:45225)
0806 16:11:03.366980 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:03.366986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:03.368614 (+ 1628us) server_negotiation.cc:407] Connection header received
0806 16:11:03.370564 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:03.370568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:03.370635 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:03.370758 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:03.372252 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.373135 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.374895 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.375232 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.375343 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.377191 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:03.377194 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:03.377197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:03.377214 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:03.377243 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:03.377251 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:03.377339 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:03.377635 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 16:11:03.377711 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 16:11:03.379633 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.367328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45225 (local address 127.8.49.2:47403)
0806 16:11:03.368363 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:03.368425 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:03.370230 (+ 1805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:03.371058 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:03.371067 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:03.371241 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:03.371995 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.372023 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.373323 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.373327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.374699 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.374713 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.375468 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.375473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.375754 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.376670 (+ 916us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:11:03.376674 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:03.376757 (+ 83us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:11:03.376889 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:03.377374 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:03.377387 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:03.377398 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 16:11:03.377584 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 16:11:03.378052 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":158,"threads_started":1}
I20260806 16:11:03.381474 984 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45225
I20260806 16:11:03.381924 984 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:03.382802 984 heartbeater.cc:511] Master 127.8.49.62:45225 requested a full tablet report, sending...
I20260806 16:11:03.384747 573 ts_manager.cc:194] Registered new tserver with Master: a8995df30e0c4221a2cb26624d005e78 (127.8.49.2:36761)
I20260806 16:11:03.385759 573 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.2:47403
W20260806 16:11:03.512172 987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:03.512490 987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:03.512531 987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:03.516767 987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:03.516855 987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:03.516917 987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:11:03.522245 987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:45225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:03.523540 987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:03.525034 987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:11:03.539098 987 server_base.cc:1034] running on GCE node
W20260806 16:11:03.539017 994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:03.539059 996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:03.539325 993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:03.540421 987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:03.541332 987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:03.544977 987 hybrid_clock.cc:648] HybridClock initialized: now 1786032663544913 us; error 72 us; skew 500 ppm
I20260806 16:11:03.548254 987 webserver.cc:460] Webserver started at http://127.8.49.3:42781/ using document root <none> and password file <none>
I20260806 16:11:03.549012 987 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:03.549099 987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:03.549351 987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:03.552316 987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "33da7e7e908e4838ab275d0b21d7225b"
format_stamp: "Formatted at 2026-08-06 16:11:03 on dist-test-slave-7z32"
server_key: "1d3fcacffe8b43b05280077b815346e7"
server_key_iv: "cb9ee7128ed8d939b64aa8af4cd4990e"
server_key_version: "encryptionkey@0"
I20260806 16:11:03.553035 987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "33da7e7e908e4838ab275d0b21d7225b"
format_stamp: "Formatted at 2026-08-06 16:11:03 on dist-test-slave-7z32"
server_key: "1d3fcacffe8b43b05280077b815346e7"
server_key_iv: "cb9ee7128ed8d939b64aa8af4cd4990e"
server_key_version: "encryptionkey@0"
I20260806 16:11:03.558621 987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:11:03.562659 1003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:03.564881 987 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:03.565397 987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "33da7e7e908e4838ab275d0b21d7225b"
format_stamp: "Formatted at 2026-08-06 16:11:03 on dist-test-slave-7z32"
server_key: "1d3fcacffe8b43b05280077b815346e7"
server_key_iv: "cb9ee7128ed8d939b64aa8af4cd4990e"
server_key_version: "encryptionkey@0"
I20260806 16:11:03.565618 987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:03.604964 987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:03.606297 987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:03.606990 987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:03.608192 987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:03.609665 987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:03.609875 987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:03.609938 987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:03.610004 987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:03.627103 987 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:33909
I20260806 16:11:03.627184 1148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:33909 every 8 connection(s)
I20260806 16:11:03.628499 987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:11:03.631443 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 987
I20260806 16:11:03.631610 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:11:03.632156 8388 external_mini_cluster.cc:1428] Setting key 3715e0e5d4a1699a78aa2d51ab796ccd
I20260806 16:11:03.641325 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.631701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:38641 (local address 127.8.49.62:45225)
0806 16:11:03.631955 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:03.631963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:03.633966 (+ 2003us) server_negotiation.cc:407] Connection header received
0806 16:11:03.636068 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:03.636071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:03.636131 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:03.636212 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:03.637218 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.637775 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.639066 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.639298 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.639419 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.640738 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:03.640742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:03.640745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:03.640769 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:03.640789 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:03.640797 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:03.640896 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:03.641081 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 16:11:03.641154 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 16:11:03.643060 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.632540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:45225 (local address 127.8.49.3:38641)
0806 16:11:03.633767 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:03.633853 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:03.635795 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:03.636424 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:03.636432 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:03.636528 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:03.637065 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.637078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.637918 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.637922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.638938 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.638949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.639489 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.639495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.639679 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.640415 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:11:03.640417 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:03.640465 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:11:03.640549 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:03.640904 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:03.640916 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:03.640923 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:11:03.641081 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 16:11:03.641656 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":221,"threads_started":1}
I20260806 16:11:03.645931 1150 heartbeater.cc:347] Connected to a master server at 127.8.49.62:45225
I20260806 16:11:03.646332 1150 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:03.647186 1150 heartbeater.cc:511] Master 127.8.49.62:45225 requested a full tablet report, sending...
I20260806 16:11:03.649252 569 ts_manager.cc:194] Registered new tserver with Master: 33da7e7e908e4838ab275d0b21d7225b (127.8.49.3:33909)
I20260806 16:11:03.650081 569 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.3:38641
I20260806 16:11:03.663398 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:11:03.673920 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.667178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34026 (local address 127.8.49.62:45225)
0806 16:11:03.667613 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:03.667619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:03.667875 (+ 256us) server_negotiation.cc:407] Connection header received
0806 16:11:03.668470 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:03.668476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:03.668554 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:03.668663 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:03.669035 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.669573 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.670802 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.671218 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.671391 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.672782 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:03.672786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:03.672789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:03.672808 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:03.672838 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:03.672849 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:03.672972 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:03.673364 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 16:11:03.673501 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260806 16:11:03.680130 569 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:11:03.684116 569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:11:03.710258 1169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.702448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33490 (local address 127.8.49.1:38827)
0806 16:11:03.702963 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:03.702971 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:03.702994 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:11:03.703074 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:03.703079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:03.703272 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:03.703431 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:03.703699 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.705017 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.707766 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.709145 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.709318 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.709610 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:03.709859 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 16:11:03.710030 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":159,"threads_started":1}
I20260806 16:11:03.710275 1168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.702099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:38827 (local address 127.0.0.1:33490)
0806 16:11:03.702523 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:03.702539 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:03.702646 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:03.703407 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:03.703411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:03.703429 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:03.703544 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.703554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.705170 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.705174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.707501 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.707520 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.709358 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.709363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.709561 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.709612 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 16:11:03.709781 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 16:11:03.710027 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":58,"thread_start_us":138,"threads_started":1}
I20260806 16:11:03.710655 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.699965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:36761 (local address 127.0.0.1:48874)
0806 16:11:03.701559 (+ 1594us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:03.701640 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:03.701851 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:03.702806 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:03.702810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:03.702857 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:03.703055 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.703063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.705313 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.705328 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.707556 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.707578 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.709742 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.709749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.709893 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.709916 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:11:03.709991 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 16:11:03.710054 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1437,"mutex_wait_us":544,"thread_start_us":559,"threads_started":1}
I20260806 16:11:03.710796 1167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.700314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48874 (local address 127.8.49.2:36761)
0806 16:11:03.700821 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:03.700847 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:03.701773 (+ 926us) server_negotiation.cc:407] Connection header received
0806 16:11:03.702130 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:03.702141 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:03.702420 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:03.702605 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:03.703312 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.704795 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.707852 (+ 3057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.709186 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.709360 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.709661 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:03.710145 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 16:11:03.710289 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":142,"threads_started":1}
I20260806 16:11:03.711252 1164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.699760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:33909 (local address 127.0.0.1:57268)
0806 16:11:03.701563 (+ 1803us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:03.701643 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:03.702058 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:03.702779 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:03.702786 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:03.702811 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:03.703271 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.703285 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.705730 (+ 2445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.705734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.707629 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.707643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.709720 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.709724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.709997 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.710035 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 16:11:03.710133 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 16:11:03.710244 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1532,"mutex_wait_us":201,"thread_start_us":184,"threads_started":1}
I20260806 16:11:03.711853 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.699931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57268 (local address 127.8.49.3:33909)
0806 16:11:03.701279 (+ 1348us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:03.701298 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:03.701921 (+ 623us) server_negotiation.cc:407] Connection header received
0806 16:11:03.702249 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:03.702257 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:03.702486 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:03.702655 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:03.703702 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.705316 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.707844 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.709148 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.709318 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.709610 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:03.710373 (+ 763us) server_negotiation.cc:299] Negotiation successful
0806 16:11:03.711674 (+ 1301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1021,"thread_start_us":297,"threads_started":1}
I20260806 16:11:03.714382 726 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddee06023304b05bc377ae953127f1d (DEFAULT_TABLE table=test-table [id=a8569c0217534d5c9872be20a08f79db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:03.714754 1065 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddee06023304b05bc377ae953127f1d (DEFAULT_TABLE table=test-table [id=a8569c0217534d5c9872be20a08f79db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:03.714984 893 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddee06023304b05bc377ae953127f1d (DEFAULT_TABLE table=test-table [id=a8569c0217534d5c9872be20a08f79db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:03.715891 726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddee06023304b05bc377ae953127f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:03.715950 1065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddee06023304b05bc377ae953127f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:03.716555 893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddee06023304b05bc377ae953127f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:03.728053 1170 tablet_bootstrap.cc:492] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: Bootstrap starting.
I20260806 16:11:03.731918 1170 tablet_bootstrap.cc:654] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:03.733209 1173 tablet_bootstrap.cc:492] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea: Bootstrap starting.
I20260806 16:11:03.733340 1170 log.cc:824] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:03.734158 1172 tablet_bootstrap.cc:492] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: Bootstrap starting.
I20260806 16:11:03.736212 1173 tablet_bootstrap.cc:654] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:03.737124 1172 tablet_bootstrap.cc:654] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:03.737519 1173 log.cc:824] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:03.738885 1172 log.cc:824] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:03.740489 1170 tablet_bootstrap.cc:492] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: No bootstrap required, opened a new log
I20260806 16:11:03.740486 1173 tablet_bootstrap.cc:492] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea: No bootstrap required, opened a new log
I20260806 16:11:03.740760 1170 ts_tablet_manager.cc:1397] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260806 16:11:03.740760 1173 ts_tablet_manager.cc:1397] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260806 16:11:03.742630 1172 tablet_bootstrap.cc:492] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: No bootstrap required, opened a new log
I20260806 16:11:03.743034 1172 ts_tablet_manager.cc:1397] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260806 16:11:03.746127 1170 raft_consensus.cc:348] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33da7e7e908e4838ab275d0b21d7225b" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 33909 } } peers { permanent_uuid: "a8995df30e0c4221a2cb26624d005e78" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36761 } } peers { permanent_uuid: "e9d99ee7640f4089986fb20670069fea" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38827 } }
I20260806 16:11:03.746515 1170 raft_consensus.cc:374] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:03.746590 1170 raft_consensus.cc:729] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8995df30e0c4221a2cb26624d005e78, State: Initialized, Role: FOLLOWER
I20260806 16:11:03.747282 1170 consensus_queue.cc:260] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33da7e7e908e4838ab275d0b21d7225b" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 33909 } } peers { permanent_uuid: "a8995df30e0c4221a2cb26624d005e78" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36761 } } peers { permanent_uuid: "e9d99ee7640f4089986fb20670069fea" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38827 } }
I20260806 16:11:03.747908 1173 raft_consensus.cc:348] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33da7e7e908e4838ab275d0b21d7225b" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 33909 } } peers { permanent_uuid: "a8995df30e0c4221a2cb26624d005e78" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36761 } } peers { permanent_uuid: "e9d99ee7640f4089986fb20670069fea" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38827 } }
I20260806 16:11:03.748219 1173 raft_consensus.cc:374] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:03.748282 1173 raft_consensus.cc:729] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d99ee7640f4089986fb20670069fea, State: Initialized, Role: FOLLOWER
I20260806 16:11:03.748615 1170 ts_tablet_manager.cc:1428] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260806 16:11:03.748811 1173 consensus_queue.cc:260] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33da7e7e908e4838ab275d0b21d7225b" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 33909 } } peers { permanent_uuid: "a8995df30e0c4221a2cb26624d005e78" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36761 } } peers { permanent_uuid: "e9d99ee7640f4089986fb20670069fea" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38827 } }
I20260806 16:11:03.749073 984 heartbeater.cc:503] Master 127.8.49.62:45225 was elected leader, sending a full tablet report...
I20260806 16:11:03.749991 1173 ts_tablet_manager.cc:1428] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea: Time spent starting tablet: real 0.009s user 0.006s sys 0.003s
I20260806 16:11:03.750205 818 heartbeater.cc:503] Master 127.8.49.62:45225 was elected leader, sending a full tablet report...
I20260806 16:11:03.750717 1172 raft_consensus.cc:348] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33da7e7e908e4838ab275d0b21d7225b" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 33909 } } peers { permanent_uuid: "a8995df30e0c4221a2cb26624d005e78" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36761 } } peers { permanent_uuid: "e9d99ee7640f4089986fb20670069fea" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38827 } }
I20260806 16:11:03.751183 1172 raft_consensus.cc:374] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:03.751271 1172 raft_consensus.cc:729] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33da7e7e908e4838ab275d0b21d7225b, State: Initialized, Role: FOLLOWER
I20260806 16:11:03.751919 1172 consensus_queue.cc:260] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33da7e7e908e4838ab275d0b21d7225b" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 33909 } } peers { permanent_uuid: "a8995df30e0c4221a2cb26624d005e78" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36761 } } peers { permanent_uuid: "e9d99ee7640f4089986fb20670069fea" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38827 } }
I20260806 16:11:03.752964 1150 heartbeater.cc:503] Master 127.8.49.62:45225 was elected leader, sending a full tablet report...
I20260806 16:11:03.754070 1172 ts_tablet_manager.cc:1428] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: Time spent starting tablet: real 0.011s user 0.008s sys 0.001s
W20260806 16:11:03.853807 819 tablet.cc:2369] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:11:03.865746 985 tablet.cc:2369] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:11:03.880062 1152 tablet.cc:2369] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:11:03.958390 1181 raft_consensus.cc:482] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:11:03.958815 1181 raft_consensus.cc:504] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33da7e7e908e4838ab275d0b21d7225b" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 33909 } } peers { permanent_uuid: "a8995df30e0c4221a2cb26624d005e78" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36761 } } peers { permanent_uuid: "e9d99ee7640f4089986fb20670069fea" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38827 } }
I20260806 16:11:03.960594 1181 leader_election.cc:288] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8995df30e0c4221a2cb26624d005e78 (127.8.49.2:36761), e9d99ee7640f4089986fb20670069fea (127.8.49.1:38827)
I20260806 16:11:03.967236 1167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.961102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:37409 (local address 127.8.49.2:36761)
0806 16:11:03.961289 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:03.961295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:03.961322 (+ 27us) server_negotiation.cc:407] Connection header received
0806 16:11:03.961416 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:03.961423 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:03.961503 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:03.961818 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:03.962217 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.963174 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.965804 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.966405 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.966572 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.966673 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:03.966961 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 16:11:03.967056 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 16:11:03.967371 1169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.961362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:45953 (local address 127.8.49.1:38827)
0806 16:11:03.961513 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:03.961519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:03.961751 (+ 232us) server_negotiation.cc:407] Connection header received
0806 16:11:03.961875 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:03.961878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:03.961936 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:03.962046 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:03.962364 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.963175 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.965865 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.966398 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.966594 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.966683 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:03.967058 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 16:11:03.967157 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 16:11:03.967236 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.960911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:36761 (local address 127.8.49.3:37409)
0806 16:11:03.961119 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:03.961141 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:03.961274 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:03.961843 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:03.961847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:03.961865 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:03.961995 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.962027 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.963367 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.963372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.965530 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.965553 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.966585 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.966590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.966722 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.966762 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 16:11:03.966935 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 16:11:03.967056 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":250}
I20260806 16:11:03.967458 1182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:03.961119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:38827 (local address 127.8.49.3:45953)
0806 16:11:03.961592 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:03.961604 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:03.961755 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:03.962076 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:03.962082 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:03.962098 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:03.962230 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.962235 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.963367 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.963372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.965530 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:03.965553 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:03.966646 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:03.966652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:03.966851 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:03.966913 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 16:11:03.967033 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 16:11:03.967146 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":190,"thread_start_us":200,"threads_started":1}
I20260806 16:11:03.969128 917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddee06023304b05bc377ae953127f1d" candidate_uuid: "33da7e7e908e4838ab275d0b21d7225b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8995df30e0c4221a2cb26624d005e78" is_pre_election: true
I20260806 16:11:03.969152 753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddee06023304b05bc377ae953127f1d" candidate_uuid: "33da7e7e908e4838ab275d0b21d7225b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d99ee7640f4089986fb20670069fea" is_pre_election: true
I20260806 16:11:03.969674 753 raft_consensus.cc:2454] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33da7e7e908e4838ab275d0b21d7225b in term 0.
I20260806 16:11:03.969666 917 raft_consensus.cc:2454] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33da7e7e908e4838ab275d0b21d7225b in term 0.
I20260806 16:11:03.970525 1004 leader_election.cc:302] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 33da7e7e908e4838ab275d0b21d7225b, a8995df30e0c4221a2cb26624d005e78, e9d99ee7640f4089986fb20670069fea; no voters:
I20260806 16:11:03.971380 1181 raft_consensus.cc:2781] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:11:03.971496 1181 raft_consensus.cc:482] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:11:03.971529 1181 raft_consensus.cc:3037] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:03.973060 1181 raft_consensus.cc:504] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33da7e7e908e4838ab275d0b21d7225b" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 33909 } } peers { permanent_uuid: "a8995df30e0c4221a2cb26624d005e78" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36761 } } peers { permanent_uuid: "e9d99ee7640f4089986fb20670069fea" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38827 } }
I20260806 16:11:03.973613 1181 leader_election.cc:288] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [CANDIDATE]: Term 1 election: Requested vote from peers a8995df30e0c4221a2cb26624d005e78 (127.8.49.2:36761), e9d99ee7640f4089986fb20670069fea (127.8.49.1:38827)
I20260806 16:11:03.974912 753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddee06023304b05bc377ae953127f1d" candidate_uuid: "33da7e7e908e4838ab275d0b21d7225b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d99ee7640f4089986fb20670069fea"
I20260806 16:11:03.974892 917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddee06023304b05bc377ae953127f1d" candidate_uuid: "33da7e7e908e4838ab275d0b21d7225b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8995df30e0c4221a2cb26624d005e78"
I20260806 16:11:03.975132 753 raft_consensus.cc:3037] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:03.975167 917 raft_consensus.cc:3037] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:03.976830 753 raft_consensus.cc:2454] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33da7e7e908e4838ab275d0b21d7225b in term 1.
I20260806 16:11:03.976899 917 raft_consensus.cc:2454] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33da7e7e908e4838ab275d0b21d7225b in term 1.
I20260806 16:11:03.977793 1009 leader_election.cc:302] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33da7e7e908e4838ab275d0b21d7225b, a8995df30e0c4221a2cb26624d005e78; no voters:
I20260806 16:11:03.978245 1181 raft_consensus.cc:2781] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:03.978669 1181 raft_consensus.cc:686] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 1 LEADER]: Becoming Leader. State: Replica: 33da7e7e908e4838ab275d0b21d7225b, State: Running, Role: LEADER
I20260806 16:11:03.979627 1181 consensus_queue.cc:237] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33da7e7e908e4838ab275d0b21d7225b" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 33909 } } peers { permanent_uuid: "a8995df30e0c4221a2cb26624d005e78" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36761 } } peers { permanent_uuid: "e9d99ee7640f4089986fb20670069fea" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38827 } }
I20260806 16:11:03.986200 567 catalog_manager.cc:5495] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b reported cstate change: term changed from 0 to 1, leader changed from <none> to 33da7e7e908e4838ab275d0b21d7225b (127.8.49.3). New cstate: current_term: 1 leader_uuid: "33da7e7e908e4838ab275d0b21d7225b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33da7e7e908e4838ab275d0b21d7225b" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8995df30e0c4221a2cb26624d005e78" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9d99ee7640f4089986fb20670069fea" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38827 } health_report { overall_health: UNKNOWN } } }
I20260806 16:11:04.020255 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:04.013933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57274 (local address 127.8.49.3:33909)
0806 16:11:04.014108 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:04.014112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:04.014267 (+ 155us) server_negotiation.cc:407] Connection header received
0806 16:11:04.014458 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:04.014461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:04.014550 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:04.014739 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:04.015286 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:04.015975 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:04.017332 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:04.017689 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:04.017857 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:04.018720 (+ 863us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:04.019504 (+ 784us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:04.019641 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:04.019935 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 16:11:04.020029 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 16:11:04.029664 753 raft_consensus.cc:1261] T 5ddee06023304b05bc377ae953127f1d P e9d99ee7640f4089986fb20670069fea [term 1 FOLLOWER]: Refusing update from remote peer 33da7e7e908e4838ab275d0b21d7225b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:04.029664 917 raft_consensus.cc:1261] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78 [term 1 FOLLOWER]: Refusing update from remote peer 33da7e7e908e4838ab275d0b21d7225b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:04.030736 1184 consensus_queue.cc:1035] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8995df30e0c4221a2cb26624d005e78" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:04.031062 1181 consensus_queue.cc:1035] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d99ee7640f4089986fb20670069fea" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 38827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:04.041064 1188 mvcc.cc:205] Tried to move back new op lower bound from 7315589791851593728 to 7315589791666331648. Current Snapshot: MvccSnapshot[applied={T|T < 7315589791851593728}]
I20260806 16:11:04.045297 1189 mvcc.cc:205] Tried to move back new op lower bound from 7315589791851593728 to 7315589791666331648. Current Snapshot: MvccSnapshot[applied={T|T < 7315589791851593728}]
I20260806 16:11:04.057279 569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:04.057616 569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:04.061691 569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bda7eeb10d0b4f9db9952fa559350403: Sending DeleteTablet for 3 replicas of tablet 5ddee06023304b05bc377ae953127f1d
I20260806 16:11:04.063432 893 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddee06023304b05bc377ae953127f1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:04 UTC) from {username='slave'} at 127.0.0.1:48874
I20260806 16:11:04.063614 726 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddee06023304b05bc377ae953127f1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:04 UTC) from {username='slave'} at 127.0.0.1:33490
I20260806 16:11:04.064633 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 648
I20260806 16:11:04.064579 1065 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddee06023304b05bc377ae953127f1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:04 UTC) from {username='slave'} at 127.0.0.1:57268
I20260806 16:11:04.064708 1202 tablet_replica.cc:330] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: stopping tablet replica
I20260806 16:11:04.065178 1202 raft_consensus.cc:2229] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:11:04.068318 1202 raft_consensus.cc:2258] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:04.071554 1202 ts_tablet_manager.cc:1905] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:04.073474 1203 tablet_replica.cc:330] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: stopping tablet replica
I20260806 16:11:04.074036 1203 raft_consensus.cc:2229] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:11:04.074568 1203 raft_consensus.cc:2258] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:04.076845 1203 ts_tablet_manager.cc:1905] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:04.078786 1202 ts_tablet_manager.cc:1918] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:04.079104 1202 log.cc:1196] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/5ddee06023304b05bc377ae953127f1d
I20260806 16:11:04.079643 1202 ts_tablet_manager.cc:1939] T 5ddee06023304b05bc377ae953127f1d P a8995df30e0c4221a2cb26624d005e78: Deleting consensus metadata
I20260806 16:11:04.080673 1203 ts_tablet_manager.cc:1918] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:04.080825 1203 log.cc:1196] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/5ddee06023304b05bc377ae953127f1d
I20260806 16:11:04.081221 1203 ts_tablet_manager.cc:1939] T 5ddee06023304b05bc377ae953127f1d P 33da7e7e908e4838ab275d0b21d7225b: Deleting consensus metadata
I20260806 16:11:04.082609 553 catalog_manager.cc:4841] TS a8995df30e0c4221a2cb26624d005e78 (127.8.49.2:36761): tablet 5ddee06023304b05bc377ae953127f1d (table test-table [id=a8569c0217534d5c9872be20a08f79db]) successfully deleted
I20260806 16:11:04.082933 550 catalog_manager.cc:4841] TS 33da7e7e908e4838ab275d0b21d7225b (127.8.49.3:33909): tablet 5ddee06023304b05bc377ae953127f1d (table test-table [id=a8569c0217534d5c9872be20a08f79db]) successfully deleted
W20260806 16:11:04.085330 550 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:38827 (error 108)
I20260806 16:11:04.085559 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 821
I20260806 16:11:04.093223 1164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:04.092610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:38827 (local address 127.0.0.1:33492)
0806 16:11:04.092968 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:04.093091 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:38827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
W20260806 16:11:04.093673 550 catalog_manager.cc:4584] TS e9d99ee7640f4089986fb20670069fea (127.8.49.1:38827): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ddee06023304b05bc377ae953127f1d: Network error: Client connection negotiation failed: client connection to 127.8.49.1:38827: connect: Connection refused (error 111)
I20260806 16:11:04.102175 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 987
I20260806 16:11:04.136445 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 532
2026-08-06T16:11:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1696 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:04 dist-test-slave-7z32 krb5kdc[1208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:04 dist-test-slave-7z32 krb5kdc[1208](info): set up 2 sockets
Aug 06 16:11:04 dist-test-slave-7z32 krb5kdc[1208](info): commencing operation
krb5kdc: starting...
W20260806 16:11:05.508639 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:05 dist-test-slave-7z32 krb5kdc[1208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:11:05.688971 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:46801
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:46801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:05.843619 1225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:05.844004 1225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:05.844054 1225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:05.849244 1225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:05.849370 1225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:05.849395 1225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:05.849414 1225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:05.849433 1225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:05.856675 1225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:46801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:46801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:05.858088 1225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:05.859330 1225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:05.869769 1235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:05.869901 1231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:05.869901 1230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:05.870606 1225 server_base.cc:1034] running on GCE node
I20260806 16:11:05.871611 1225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:05.873093 1225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:05.874526 1225 hybrid_clock.cc:648] HybridClock initialized: now 1786032665874485 us; error 69 us; skew 500 ppm
Aug 06 16:11:05 dist-test-slave-7z32 krb5kdc[1208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032665, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:05.879294 1225 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:11:05.881054 1225 webserver.cc:460] Webserver started at http://127.8.49.62:36243/ using document root <none> and password file <none>
I20260806 16:11:05.881829 1225 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:05.881960 1225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:05.882246 1225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:05.884815 1225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "beeaf0b564bb40029e3d1999630ae21d"
format_stamp: "Formatted at 2026-08-06 16:11:05 on dist-test-slave-7z32"
server_key: "ac52d957552621b1a85af988ca5f57c2"
server_key_iv: "a0fb5cf06032de272485a839c693dd27"
server_key_version: "encryptionkey@0"
I20260806 16:11:05.885509 1225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "beeaf0b564bb40029e3d1999630ae21d"
format_stamp: "Formatted at 2026-08-06 16:11:05 on dist-test-slave-7z32"
server_key: "ac52d957552621b1a85af988ca5f57c2"
server_key_iv: "a0fb5cf06032de272485a839c693dd27"
server_key_version: "encryptionkey@0"
I20260806 16:11:05.891291 1225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:11:05.896421 1243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:05.898622 1225 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:11:05.898921 1225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "beeaf0b564bb40029e3d1999630ae21d"
format_stamp: "Formatted at 2026-08-06 16:11:05 on dist-test-slave-7z32"
server_key: "ac52d957552621b1a85af988ca5f57c2"
server_key_iv: "a0fb5cf06032de272485a839c693dd27"
server_key_version: "encryptionkey@0"
I20260806 16:11:05.899142 1225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:05.917718 1225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:05.921945 1225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:05.922238 1225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:05.932186 1225 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:46801
I20260806 16:11:05.932232 1311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:46801 every 8 connection(s)
I20260806 16:11:05.933679 1225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:05.938524 1312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:05.942570 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 1225
I20260806 16:11:05.942785 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:05.943454 8388 external_mini_cluster.cc:1428] Setting key 8678f37d7f0c0b9b8270d3a2e0757de8
I20260806 16:11:05.947654 1312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d: Bootstrap starting.
I20260806 16:11:05.950858 1312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:05.951934 1312 log.cc:824] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:05.955972 1312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d: No bootstrap required, opened a new log
Aug 06 16:11:05 dist-test-slave-7z32 krb5kdc[1208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:05.960906 1312 raft_consensus.cc:348] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beeaf0b564bb40029e3d1999630ae21d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46801 } }
I20260806 16:11:05.961261 1312 raft_consensus.cc:374] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:05.961377 1312 raft_consensus.cc:729] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beeaf0b564bb40029e3d1999630ae21d, State: Initialized, Role: FOLLOWER
I20260806 16:11:05.962411 1312 consensus_queue.cc:260] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beeaf0b564bb40029e3d1999630ae21d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46801 } }
I20260806 16:11:05.962714 1312 raft_consensus.cc:388] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:05.962848 1312 raft_consensus.cc:482] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:05.963014 1312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:05.964617 1312 raft_consensus.cc:504] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beeaf0b564bb40029e3d1999630ae21d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46801 } }
I20260806 16:11:05.964670 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:05.945432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55772 (local address 127.8.49.62:46801)
0806 16:11:05.946226 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:05.946251 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:05.946310 (+ 59us) server_negotiation.cc:407] Connection header received
0806 16:11:05.947411 (+ 1101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:05.947512 (+ 101us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:05.948042 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:05.948483 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:05.949495 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:05.950683 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:05.951799 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:05.952502 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:05.952714 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:05.957142 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:05.957149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:05.957171 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:05.957215 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:05.960522 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:05.961166 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:05.961172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:05.961178 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:05.961392 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:05.961789 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:05.961793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:05.961795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:05.962391 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:05.962702 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:05.963168 (+ 466us) server_negotiation.cc:299] Negotiation successful
0806 16:11:05.963541 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":211,"threads_started":1}
I20260806 16:11:05.965432 1312 leader_election.cc:302] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beeaf0b564bb40029e3d1999630ae21d; no voters:
I20260806 16:11:05.965982 1312 leader_election.cc:288] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:05.966271 1320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:05.966648 1320 raft_consensus.cc:686] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [term 1 LEADER]: Becoming Leader. State: Replica: beeaf0b564bb40029e3d1999630ae21d, State: Running, Role: LEADER
I20260806 16:11:05.967124 1320 consensus_queue.cc:237] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beeaf0b564bb40029e3d1999630ae21d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46801 } }
I20260806 16:11:05.967857 1312 sys_catalog.cc:564] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:05.969337 1322 sys_catalog.cc:455] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beeaf0b564bb40029e3d1999630ae21d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beeaf0b564bb40029e3d1999630ae21d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46801 } } }
I20260806 16:11:05.969544 1322 sys_catalog.cc:458] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:05.969957 1323 sys_catalog.cc:455] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [sys.catalog]: SysCatalogTable state changed. Reason: New leader beeaf0b564bb40029e3d1999630ae21d. Latest consensus state: current_term: 1 leader_uuid: "beeaf0b564bb40029e3d1999630ae21d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beeaf0b564bb40029e3d1999630ae21d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 46801 } } }
I20260806 16:11:05.970074 1323 sys_catalog.cc:458] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:05.970203 1334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:05.974225 1334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:05.981841 1334 catalog_manager.cc:1347] Generated new cluster ID: 3da300b68b574825b44cf1868886f08f
I20260806 16:11:05.982074 1334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:06.006815 1334 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:06.008140 1334 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:06.022210 1334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d: Generated new TSK 0
I20260806 16:11:06.023883 1334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:11:06.069732 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46801
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:06.211509 1347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:06.211884 1347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:06.211966 1347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:06.216922 1347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:06.217090 1347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:06.217202 1347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:11:06.223335 1347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:06.224637 1347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:06.225798 1347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:06.238633 1353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:06.238646 1356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:06.239255 1347 server_base.cc:1034] running on GCE node
W20260806 16:11:06.239306 1354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:06.239804 1347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:06.240624 1347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:06.242051 1347 hybrid_clock.cc:648] HybridClock initialized: now 1786032666242006 us; error 57 us; skew 500 ppm
Aug 06 16:11:06 dist-test-slave-7z32 krb5kdc[1208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:06.259199 1347 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:11:06.260464 1347 webserver.cc:460] Webserver started at http://127.8.49.1:46267/ using document root <none> and password file <none>
I20260806 16:11:06.261210 1347 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:06.261296 1347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:06.261577 1347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:06.264099 1347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "3555987774394799abaaf5403263b07d"
format_stamp: "Formatted at 2026-08-06 16:11:06 on dist-test-slave-7z32"
server_key: "355a6ed0926ecbd7c2dbee1c90d03da1"
server_key_iv: "a20bdef1d9449fcbb4533a700da81f44"
server_key_version: "encryptionkey@0"
I20260806 16:11:06.264742 1347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "3555987774394799abaaf5403263b07d"
format_stamp: "Formatted at 2026-08-06 16:11:06 on dist-test-slave-7z32"
server_key: "355a6ed0926ecbd7c2dbee1c90d03da1"
server_key_iv: "a20bdef1d9449fcbb4533a700da81f44"
server_key_version: "encryptionkey@0"
I20260806 16:11:06.269731 1347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:11:06.273451 1363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:06.275520 1347 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:11:06.275736 1347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "3555987774394799abaaf5403263b07d"
format_stamp: "Formatted at 2026-08-06 16:11:06 on dist-test-slave-7z32"
server_key: "355a6ed0926ecbd7c2dbee1c90d03da1"
server_key_iv: "a20bdef1d9449fcbb4533a700da81f44"
server_key_version: "encryptionkey@0"
I20260806 16:11:06.275890 1347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:06.323747 1347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:06.328541 1347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:06.328832 1347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:06.329952 1347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:06.331179 1347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:06.331271 1347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:06.331362 1347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:06.331416 1347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:06.345373 1347 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:34655
I20260806 16:11:06.345397 1503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:34655 every 8 connection(s)
I20260806 16:11:06.346465 1347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:11:06.353955 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 1347
I20260806 16:11:06.354370 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:11:06.355003 8388 external_mini_cluster.cc:1428] Setting key 1f7044fab844e1fde8f1c436bafa178b
Aug 06 16:11:06 dist-test-slave-7z32 krb5kdc[1208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:06.368448 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:06.349256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:36743 (local address 127.8.49.62:46801)
0806 16:11:06.349505 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:06.349510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:06.350737 (+ 1227us) server_negotiation.cc:407] Connection header received
0806 16:11:06.352169 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:06.352174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:06.352252 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:06.352404 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:06.354273 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.355056 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.357105 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.357486 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.357593 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:06.362200 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:06.362206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:06.362208 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:06.362258 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:06.364033 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:06.365388 (+ 1355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:06.365395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:06.365407 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:06.365529 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:06.366395 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:06.366399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:06.366400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:06.366685 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:06.366793 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:06.367982 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0806 16:11:06.368198 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 16:11:06.370422 1508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:06.349700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46801 (local address 127.8.49.1:36743)
0806 16:11:06.350460 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:06.350564 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:06.351762 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:06.352648 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:06.352657 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:06.353139 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:06.354056 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:06.354074 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.355379 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.355386 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:06.356896 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:06.356921 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.357727 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.357732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:06.358039 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:06.359135 (+ 1096us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:06.359171 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:06.361883 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:06.364247 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:06.364262 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:06.364293 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:06.365156 (+ 863us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:06.365695 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:06.365701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:06.365706 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:06.366199 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:06.366883 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:06.366895 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:06.367532 (+ 637us) client_negotiation.cc:769] Sending connection context
0806 16:11:06.367895 (+ 363us) client_negotiation.cc:240] Negotiation successful
0806 16:11:06.368612 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":202,"threads_started":1}
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
I20260806 16:11:06.372458 1506 heartbeater.cc:347] Connected to a master server at 127.8.49.62:46801
I20260806 16:11:06.372794 1506 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:06.373605 1506 heartbeater.cc:511] Master 127.8.49.62:46801 requested a full tablet report, sending...
I20260806 16:11:06.376367 1264 ts_manager.cc:194] Registered new tserver with Master: 3555987774394799abaaf5403263b07d (127.8.49.1:34655)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:11:06.378149 1264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:36743
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:11:06.406106 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46801
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:06.572386 1511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:06.572896 1511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:06.572959 1511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:06.577836 1511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:06.577957 1511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:06.578032 1511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:11:06.586082 1511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:06.587881 1511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:06.590497 1511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:06.606101 1517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:06.606104 1524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:06.606104 1522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:06.606851 1511 server_base.cc:1034] running on GCE node
I20260806 16:11:06.607484 1511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:06.608461 1511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:06.609669 1511 hybrid_clock.cc:648] HybridClock initialized: now 1786032666609645 us; error 45 us; skew 500 ppm
Aug 06 16:11:06 dist-test-slave-7z32 krb5kdc[1208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:06.613709 1511 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:11:06.615051 1511 webserver.cc:460] Webserver started at http://127.8.49.2:44523/ using document root <none> and password file <none>
I20260806 16:11:06.616045 1511 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:06.616159 1511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:06.616389 1511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:06.619196 1511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "81b904039b804de8ab176709db84351c"
format_stamp: "Formatted at 2026-08-06 16:11:06 on dist-test-slave-7z32"
server_key: "1726c32ac5796816ab8961b5f4c18005"
server_key_iv: "36ea992471aa817c135f6d6f741ce550"
server_key_version: "encryptionkey@0"
I20260806 16:11:06.619879 1511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "81b904039b804de8ab176709db84351c"
format_stamp: "Formatted at 2026-08-06 16:11:06 on dist-test-slave-7z32"
server_key: "1726c32ac5796816ab8961b5f4c18005"
server_key_iv: "36ea992471aa817c135f6d6f741ce550"
server_key_version: "encryptionkey@0"
I20260806 16:11:06.624949 1511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:11:06.628790 1535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:06.630929 1511 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:11:06.631106 1511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "81b904039b804de8ab176709db84351c"
format_stamp: "Formatted at 2026-08-06 16:11:06 on dist-test-slave-7z32"
server_key: "1726c32ac5796816ab8961b5f4c18005"
server_key_iv: "36ea992471aa817c135f6d6f741ce550"
server_key_version: "encryptionkey@0"
I20260806 16:11:06.631240 1511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:06.652537 1511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:06.656576 1511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:06.656775 1511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:06.657451 1511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:06.658628 1511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:06.658689 1511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:06.658771 1511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:06.658815 1511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:06.671861 1511 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:32867
I20260806 16:11:06.671968 1679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:32867 every 8 connection(s)
I20260806 16:11:06.673017 1511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:11:06.682350 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 1511
I20260806 16:11:06.682538 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:11:06.682969 8388 external_mini_cluster.cc:1428] Setting key 3d0ce900ef53423c81a34b9fdeebaa2f
Aug 06 16:11:06 dist-test-slave-7z32 krb5kdc[1208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:06.694375 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:06.675913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:53317 (local address 127.8.49.62:46801)
0806 16:11:06.676153 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:06.676158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:06.677247 (+ 1089us) server_negotiation.cc:407] Connection header received
0806 16:11:06.679082 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:06.679086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:06.679168 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:06.679289 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:06.681630 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.682402 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.684329 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.684598 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.684728 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:06.688701 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:06.688705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:06.688708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:06.688739 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:06.690917 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:06.691795 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:06.691805 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:06.691810 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:06.692011 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:06.692566 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:06.692572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:06.692574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:06.692959 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:06.693109 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:06.693856 (+ 747us) server_negotiation.cc:299] Negotiation successful
0806 16:11:06.694082 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 16:11:06.695195 1683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:06.676273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46801 (local address 127.8.49.2:53317)
0806 16:11:06.677007 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:06.677074 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:06.678688 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:06.679776 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:06.679787 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:06.680549 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:06.681387 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:06.681410 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.682642 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.682646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:06.684161 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:06.684180 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.684794 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.684799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:06.685037 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:06.685839 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:06.685868 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:06.688457 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:06.691104 (+ 2647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:06.691111 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:06.691131 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:06.691556 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:06.692222 (+ 666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:06.692225 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:06.692226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:06.692402 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:06.693154 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:06.693161 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:06.693496 (+ 335us) client_negotiation.cc:769] Sending connection context
0806 16:11:06.693710 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 16:11:06.694065 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":158,"threads_started":1}
I20260806 16:11:06.696870 1680 heartbeater.cc:347] Connected to a master server at 127.8.49.62:46801
I20260806 16:11:06.697315 1680 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:11:06.698200 1680 heartbeater.cc:511] Master 127.8.49.62:46801 requested a full tablet report, sending...
I20260806 16:11:06.700160 1264 ts_manager.cc:194] Registered new tserver with Master: 81b904039b804de8ab176709db84351c (127.8.49.2:32867)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:11:06.701025 1264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:53317
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:11:06.723780 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46801
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:06.883333 1687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:06.883759 1687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:06.883852 1687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:06.888556 1687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:06.888684 1687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:06.888803 1687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:11:06.895465 1687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:46801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:06.897147 1687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:06.898579 1687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:06.913352 1693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:06.913370 1695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:06.913384 1692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:06.913904 1687 server_base.cc:1034] running on GCE node
I20260806 16:11:06.914693 1687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:06.915629 1687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:06.916903 1687 hybrid_clock.cc:648] HybridClock initialized: now 1786032666916869 us; error 55 us; skew 500 ppm
Aug 06 16:11:06 dist-test-slave-7z32 krb5kdc[1208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:06.921312 1687 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:11:06.922741 1687 webserver.cc:460] Webserver started at http://127.8.49.3:40599/ using document root <none> and password file <none>
I20260806 16:11:06.923624 1687 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:06.923713 1687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:06.923952 1687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:06.927074 1687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "d458b03f8c064c8a81e77c88aa271286"
format_stamp: "Formatted at 2026-08-06 16:11:06 on dist-test-slave-7z32"
server_key: "ceef509918a38e35cb62bd2c4015ba2f"
server_key_iv: "cc5229f1696a2cd5178e8443e0587752"
server_key_version: "encryptionkey@0"
I20260806 16:11:06.927899 1687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "d458b03f8c064c8a81e77c88aa271286"
format_stamp: "Formatted at 2026-08-06 16:11:06 on dist-test-slave-7z32"
server_key: "ceef509918a38e35cb62bd2c4015ba2f"
server_key_iv: "cc5229f1696a2cd5178e8443e0587752"
server_key_version: "encryptionkey@0"
I20260806 16:11:06.933475 1687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:11:06.937222 1704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:06.939365 1687 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:06.939612 1687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "d458b03f8c064c8a81e77c88aa271286"
format_stamp: "Formatted at 2026-08-06 16:11:06 on dist-test-slave-7z32"
server_key: "ceef509918a38e35cb62bd2c4015ba2f"
server_key_iv: "cc5229f1696a2cd5178e8443e0587752"
server_key_version: "encryptionkey@0"
I20260806 16:11:06.939780 1687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:06.964540 1687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:06.968881 1687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:06.969125 1687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:06.970012 1687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:06.971217 1687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:06.971298 1687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:06.971380 1687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:06.971426 1687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:06.984697 1687 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:39103
I20260806 16:11:06.984719 1848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:39103 every 8 connection(s)
I20260806 16:11:06.986191 1687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:11:06.995939 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 1687
I20260806 16:11:06.996151 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:11:06.996503 8388 external_mini_cluster.cc:1428] Setting key e4c57ab33289a41fe14897066a3f9005
Aug 06 16:11:06 dist-test-slave-7z32 krb5kdc[1208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032666, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:07.009256 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:06.989076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:40319 (local address 127.8.49.62:46801)
0806 16:11:06.989412 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:06.989416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:06.990571 (+ 1155us) server_negotiation.cc:407] Connection header received
0806 16:11:06.992815 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:06.992819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:06.992904 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:06.993035 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:06.995360 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.996020 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.997869 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.998148 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.998314 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.003186 (+ 4872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:07.003190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:07.003193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:07.003235 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:07.005156 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:07.006303 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:07.006307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:07.006308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:07.006417 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:07.007398 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:07.007408 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:07.007410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:07.007721 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:07.007894 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:07.008813 (+ 919us) server_negotiation.cc:299] Negotiation successful
0806 16:11:07.009024 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260806 16:11:07.010694 1852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:06.989498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:46801 (local address 127.8.49.3:40319)
0806 16:11:06.990294 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:06.990370 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:06.992450 (+ 2080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:06.993463 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:06.993477 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:06.994204 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:06.995091 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:06.995117 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.996200 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.996206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:06.997697 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:06.997711 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:06.998417 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:06.998422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:06.998654 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.000061 (+ 1407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:07.000094 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:07.002791 (+ 2697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:07.005456 (+ 2665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:07.005470 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:07.005502 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:07.006095 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:07.006545 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:07.006549 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:07.006551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:07.007228 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:07.007874 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:07.007886 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:07.008433 (+ 547us) client_negotiation.cc:769] Sending connection context
0806 16:11:07.008741 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 16:11:07.009307 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":164,"threads_started":1}
I20260806 16:11:07.012823 1849 heartbeater.cc:347] Connected to a master server at 127.8.49.62:46801
I20260806 16:11:07.013458 1849 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:07.014271 1849 heartbeater.cc:511] Master 127.8.49.62:46801 requested a full tablet report, sending...
I20260806 16:11:07.016029 1264 ts_manager.cc:194] Registered new tserver with Master: d458b03f8c064c8a81e77c88aa271286 (127.8.49.3:39103)
I20260806 16:11:07.016904 1264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:40319
I20260806 16:11:07.025882 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:11:07.041772 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.029611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55778 (local address 127.8.49.62:46801)
0806 16:11:07.029825 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:07.029829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:07.030049 (+ 220us) server_negotiation.cc:407] Connection header received
0806 16:11:07.030137 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:07.030140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:07.030220 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:07.030340 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:07.031445 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.032053 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.033161 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.033421 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.033531 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.036258 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:07.036263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:07.036267 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:07.036338 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:07.038126 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:07.039398 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:07.039403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:07.039404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:07.039519 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:07.040248 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:07.040252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:07.040253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:07.040582 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:07.040742 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:07.041263 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 16:11:07.041482 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 16:11:07.049386 1264 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:11:07.054447 1264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:11:07.077131 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.069137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:32867 (local address 127.0.0.1:42528)
0806 16:11:07.070455 (+ 1318us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:07.070472 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:07.070597 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:07.071271 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:07.071274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:07.071294 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:07.071456 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.071466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.073324 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.073327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.074648 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.074663 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.076313 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.076322 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.076485 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.076552 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 16:11:07.076717 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 16:11:07.076881 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1221,"thread_start_us":129,"threads_started":1}
I20260806 16:11:07.077543 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.068231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:39103 (local address 127.0.0.1:33380)
0806 16:11:07.069465 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:07.069558 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:07.069775 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:07.070932 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:07.070940 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:07.070972 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:07.071203 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.071213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.073175 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.073180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.075299 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.075316 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.076985 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.076989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.077136 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.077211 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 16:11:07.077296 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 16:11:07.077376 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"mutex_wait_us":389,"thread_start_us":166,"threads_started":1}
I20260806 16:11:07.077831 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.068856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57484 (local address 127.8.49.1:34655)
0806 16:11:07.069362 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:07.069373 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:07.070263 (+ 890us) server_negotiation.cc:407] Connection header received
0806 16:11:07.070562 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:07.070568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:07.070764 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:07.070959 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:07.071575 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.072831 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.075573 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.076486 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.076615 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.076756 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:07.077439 (+ 683us) server_negotiation.cc:299] Negotiation successful
0806 16:11:07.077596 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":88,"threads_started":1}
I20260806 16:11:07.078092 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.069706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42528 (local address 127.8.49.2:32867)
0806 16:11:07.070234 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:07.070250 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:07.070717 (+ 467us) server_negotiation.cc:407] Connection header received
0806 16:11:07.070805 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:07.070810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:07.071116 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:07.071298 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:07.071690 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.073126 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.074876 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.076102 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.076265 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.076515 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:07.077599 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0806 16:11:07.077771 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":173,"threads_started":1}
I20260806 16:11:07.078346 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.068526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33380 (local address 127.8.49.3:39103)
0806 16:11:07.069747 (+ 1221us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:07.069766 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:07.069803 (+ 37us) server_negotiation.cc:407] Connection header received
0806 16:11:07.070048 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:07.070057 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:07.070382 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:07.070581 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:07.071372 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.072946 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.075545 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.076625 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.076741 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.076906 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:07.077848 (+ 942us) server_negotiation.cc:299] Negotiation successful
0806 16:11:07.078133 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":990,"thread_start_us":111,"threads_started":1}
I20260806 16:11:07.078761 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.068502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:34655 (local address 127.0.0.1:57484)
0806 16:11:07.070133 (+ 1631us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:07.070150 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:07.070352 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:07.071179 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:07.071182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:07.071201 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:07.071347 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.071355 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.073050 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.073058 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.075297 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.075316 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.076928 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.076933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.077073 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.077122 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 16:11:07.078268 (+ 1146us) client_negotiation.cc:240] Negotiation successful
0806 16:11:07.078584 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1555,"mutex_wait_us":338,"thread_start_us":272,"threads_started":1}
I20260806 16:11:07.082491 1595 tablet_service.cc:1467] Processing CreateTablet for tablet 52a40fb55bca4a66bbee087244572cb5 (DEFAULT_TABLE table=test-table [id=e72b1fbda77741b094a4207d7474f222]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:07.082793 1765 tablet_service.cc:1467] Processing CreateTablet for tablet 52a40fb55bca4a66bbee087244572cb5 (DEFAULT_TABLE table=test-table [id=e72b1fbda77741b094a4207d7474f222]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:07.082902 1425 tablet_service.cc:1467] Processing CreateTablet for tablet 52a40fb55bca4a66bbee087244572cb5 (DEFAULT_TABLE table=test-table [id=e72b1fbda77741b094a4207d7474f222]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:07.084242 1765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52a40fb55bca4a66bbee087244572cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:07.084242 1595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52a40fb55bca4a66bbee087244572cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:07.084460 1425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52a40fb55bca4a66bbee087244572cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:07.094429 1871 tablet_bootstrap.cc:492] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: Bootstrap starting.
I20260806 16:11:07.096275 1873 tablet_bootstrap.cc:492] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d: Bootstrap starting.
I20260806 16:11:07.098373 1871 tablet_bootstrap.cc:654] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:07.098996 1872 tablet_bootstrap.cc:492] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286: Bootstrap starting.
I20260806 16:11:07.099242 1873 tablet_bootstrap.cc:654] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:07.099715 1871 log.cc:824] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:07.100425 1873 log.cc:824] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:07.102581 1871 tablet_bootstrap.cc:492] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: No bootstrap required, opened a new log
I20260806 16:11:07.102701 1872 tablet_bootstrap.cc:654] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:07.102909 1871 ts_tablet_manager.cc:1397] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260806 16:11:07.104048 1873 tablet_bootstrap.cc:492] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d: No bootstrap required, opened a new log
I20260806 16:11:07.104351 1873 ts_tablet_manager.cc:1397] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:11:07.104898 1872 log.cc:824] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:07.107704 1871 raft_consensus.cc:348] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d458b03f8c064c8a81e77c88aa271286" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 39103 } } peers { permanent_uuid: "3555987774394799abaaf5403263b07d" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34655 } } peers { permanent_uuid: "81b904039b804de8ab176709db84351c" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 32867 } }
I20260806 16:11:07.108134 1871 raft_consensus.cc:374] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:07.108201 1871 raft_consensus.cc:729] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81b904039b804de8ab176709db84351c, State: Initialized, Role: FOLLOWER
I20260806 16:11:07.108537 1872 tablet_bootstrap.cc:492] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286: No bootstrap required, opened a new log
I20260806 16:11:07.108819 1872 ts_tablet_manager.cc:1397] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 16:11:07.108798 1871 consensus_queue.cc:260] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d458b03f8c064c8a81e77c88aa271286" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 39103 } } peers { permanent_uuid: "3555987774394799abaaf5403263b07d" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34655 } } peers { permanent_uuid: "81b904039b804de8ab176709db84351c" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 32867 } }
I20260806 16:11:07.109711 1680 heartbeater.cc:503] Master 127.8.49.62:46801 was elected leader, sending a full tablet report...
I20260806 16:11:07.110208 1871 ts_tablet_manager.cc:1428] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 16:11:07.110765 1873 raft_consensus.cc:348] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d458b03f8c064c8a81e77c88aa271286" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 39103 } } peers { permanent_uuid: "3555987774394799abaaf5403263b07d" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34655 } } peers { permanent_uuid: "81b904039b804de8ab176709db84351c" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 32867 } }
I20260806 16:11:07.111485 1873 raft_consensus.cc:374] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:07.111568 1873 raft_consensus.cc:729] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3555987774394799abaaf5403263b07d, State: Initialized, Role: FOLLOWER
I20260806 16:11:07.112301 1873 consensus_queue.cc:260] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d458b03f8c064c8a81e77c88aa271286" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 39103 } } peers { permanent_uuid: "3555987774394799abaaf5403263b07d" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34655 } } peers { permanent_uuid: "81b904039b804de8ab176709db84351c" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 32867 } }
I20260806 16:11:07.113415 1506 heartbeater.cc:503] Master 127.8.49.62:46801 was elected leader, sending a full tablet report...
I20260806 16:11:07.113896 1873 ts_tablet_manager.cc:1428] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d: Time spent starting tablet: real 0.009s user 0.001s sys 0.008s
I20260806 16:11:07.114074 1872 raft_consensus.cc:348] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d458b03f8c064c8a81e77c88aa271286" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 39103 } } peers { permanent_uuid: "3555987774394799abaaf5403263b07d" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34655 } } peers { permanent_uuid: "81b904039b804de8ab176709db84351c" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 32867 } }
I20260806 16:11:07.114495 1872 raft_consensus.cc:374] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:07.114589 1872 raft_consensus.cc:729] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d458b03f8c064c8a81e77c88aa271286, State: Initialized, Role: FOLLOWER
I20260806 16:11:07.115259 1872 consensus_queue.cc:260] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d458b03f8c064c8a81e77c88aa271286" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 39103 } } peers { permanent_uuid: "3555987774394799abaaf5403263b07d" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34655 } } peers { permanent_uuid: "81b904039b804de8ab176709db84351c" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 32867 } }
I20260806 16:11:07.116230 1849 heartbeater.cc:503] Master 127.8.49.62:46801 was elected leader, sending a full tablet report...
I20260806 16:11:07.116750 1872 ts_tablet_manager.cc:1428] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260806 16:11:07.150591 1877 raft_consensus.cc:482] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:11:07.150947 1877 raft_consensus.cc:504] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d458b03f8c064c8a81e77c88aa271286" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 39103 } } peers { permanent_uuid: "3555987774394799abaaf5403263b07d" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34655 } } peers { permanent_uuid: "81b904039b804de8ab176709db84351c" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 32867 } }
I20260806 16:11:07.152385 1877 leader_election.cc:288] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d458b03f8c064c8a81e77c88aa271286 (127.8.49.3:39103), 3555987774394799abaaf5403263b07d (127.8.49.1:34655)
I20260806 16:11:07.157176 1683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.152887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:39103 (local address 127.8.49.2:56915)
0806 16:11:07.153227 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:07.153248 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:07.153439 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:07.153789 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:07.153792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:07.153812 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:07.154010 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.154017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.154885 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.154889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.155807 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.155818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.156588 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.156592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.156725 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.156758 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 16:11:07.156857 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 16:11:07.156965 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":28}
I20260806 16:11:07.157163 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.153040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:56915 (local address 127.8.49.3:39103)
0806 16:11:07.153204 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:07.153209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:07.153374 (+ 165us) server_negotiation.cc:407] Connection header received
0806 16:11:07.153604 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:07.153608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:07.153664 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:07.153762 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:07.154141 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.154707 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.155954 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.156437 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.156574 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.156664 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:07.156877 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 16:11:07.156966 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 16:11:07.157187 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.153040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:56757 (local address 127.8.49.1:34655)
0806 16:11:07.153340 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:07.153345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:07.153536 (+ 191us) server_negotiation.cc:407] Connection header received
0806 16:11:07.153708 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:07.153711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:07.153768 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:07.153861 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:07.154269 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.154812 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.156057 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.156524 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.156652 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.156730 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:07.156950 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 16:11:07.157036 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260806 16:11:07.157226 1881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.152887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:34655 (local address 127.8.49.2:56757)
0806 16:11:07.153316 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:07.153337 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:07.153552 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:07.153882 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:07.153885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:07.153904 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:07.154138 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.154146 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.155001 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.155005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.155910 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.155920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.156677 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.156681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.156799 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.156823 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:11:07.156917 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 16:11:07.157030 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":61,"thread_start_us":132,"threads_started":1}
I20260806 16:11:07.158298 1790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a40fb55bca4a66bbee087244572cb5" candidate_uuid: "81b904039b804de8ab176709db84351c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d458b03f8c064c8a81e77c88aa271286" is_pre_election: true
I20260806 16:11:07.158339 1449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a40fb55bca4a66bbee087244572cb5" candidate_uuid: "81b904039b804de8ab176709db84351c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3555987774394799abaaf5403263b07d" is_pre_election: true
I20260806 16:11:07.158751 1449 raft_consensus.cc:2454] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81b904039b804de8ab176709db84351c in term 0.
I20260806 16:11:07.158771 1790 raft_consensus.cc:2454] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81b904039b804de8ab176709db84351c in term 0.
I20260806 16:11:07.159514 1536 leader_election.cc:302] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3555987774394799abaaf5403263b07d, 81b904039b804de8ab176709db84351c, d458b03f8c064c8a81e77c88aa271286; no voters:
I20260806 16:11:07.159940 1877 raft_consensus.cc:2781] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:11:07.160100 1877 raft_consensus.cc:482] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:11:07.160140 1877 raft_consensus.cc:3037] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:07.161335 1877 raft_consensus.cc:504] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d458b03f8c064c8a81e77c88aa271286" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 39103 } } peers { permanent_uuid: "3555987774394799abaaf5403263b07d" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34655 } } peers { permanent_uuid: "81b904039b804de8ab176709db84351c" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 32867 } }
I20260806 16:11:07.161796 1877 leader_election.cc:288] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [CANDIDATE]: Term 1 election: Requested vote from peers d458b03f8c064c8a81e77c88aa271286 (127.8.49.3:39103), 3555987774394799abaaf5403263b07d (127.8.49.1:34655)
I20260806 16:11:07.162341 1449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a40fb55bca4a66bbee087244572cb5" candidate_uuid: "81b904039b804de8ab176709db84351c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3555987774394799abaaf5403263b07d"
I20260806 16:11:07.162390 1790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a40fb55bca4a66bbee087244572cb5" candidate_uuid: "81b904039b804de8ab176709db84351c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d458b03f8c064c8a81e77c88aa271286"
I20260806 16:11:07.162463 1449 raft_consensus.cc:3037] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:07.162635 1790 raft_consensus.cc:3037] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:07.163883 1449 raft_consensus.cc:2454] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81b904039b804de8ab176709db84351c in term 1.
I20260806 16:11:07.164286 1790 raft_consensus.cc:2454] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81b904039b804de8ab176709db84351c in term 1.
I20260806 16:11:07.164314 1536 leader_election.cc:302] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3555987774394799abaaf5403263b07d, 81b904039b804de8ab176709db84351c; no voters:
I20260806 16:11:07.164528 1877 raft_consensus.cc:2781] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:07.164791 1877 raft_consensus.cc:686] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 1 LEADER]: Becoming Leader. State: Replica: 81b904039b804de8ab176709db84351c, State: Running, Role: LEADER
I20260806 16:11:07.165189 1877 consensus_queue.cc:237] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d458b03f8c064c8a81e77c88aa271286" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 39103 } } peers { permanent_uuid: "3555987774394799abaaf5403263b07d" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34655 } } peers { permanent_uuid: "81b904039b804de8ab176709db84351c" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 32867 } }
I20260806 16:11:07.170231 1264 catalog_manager.cc:5495] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c reported cstate change: term changed from 0 to 1, leader changed from <none> to 81b904039b804de8ab176709db84351c (127.8.49.2). New cstate: current_term: 1 leader_uuid: "81b904039b804de8ab176709db84351c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d458b03f8c064c8a81e77c88aa271286" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 39103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3555987774394799abaaf5403263b07d" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81b904039b804de8ab176709db84351c" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 32867 } health_report { overall_health: HEALTHY } } }
W20260806 16:11:07.174750 1681 tablet.cc:2369] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:11:07.193023 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.188513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42534 (local address 127.8.49.2:32867)
0806 16:11:07.188725 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:07.188731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:07.188750 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:11:07.188956 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:07.188959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:07.189027 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:07.189185 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:07.189504 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.190095 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.191184 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.191440 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.191571 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.191967 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:07.192477 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:07.192660 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:07.192773 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 16:11:07.192820 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 16:11:07.201817 1449 raft_consensus.cc:1261] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d [term 1 FOLLOWER]: Refusing update from remote peer 81b904039b804de8ab176709db84351c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:07.201956 1790 raft_consensus.cc:1261] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286 [term 1 FOLLOWER]: Refusing update from remote peer 81b904039b804de8ab176709db84351c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:07.202926 1884 consensus_queue.cc:1035] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3555987774394799abaaf5403263b07d" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 34655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:07.203207 1877 consensus_queue.cc:1035] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d458b03f8c064c8a81e77c88aa271286" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 39103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:07.215621 1895 mvcc.cc:205] Tried to move back new op lower bound from 7315589804847824896 to 7315589804711862272. Current Snapshot: MvccSnapshot[applied={T|T < 7315589804847824896}]
I20260806 16:11:07.229532 1262 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:07.229827 1262 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:07.233636 1262 catalog_manager.cc:5782] T 00000000000000000000000000000000 P beeaf0b564bb40029e3d1999630ae21d: Sending DeleteTablet for 3 replicas of tablet 52a40fb55bca4a66bbee087244572cb5
I20260806 16:11:07.235078 1595 tablet_service.cc:1514] Processing DeleteTablet for tablet 52a40fb55bca4a66bbee087244572cb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:07 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:42528
I20260806 16:11:07.235106 1765 tablet_service.cc:1514] Processing DeleteTablet for tablet 52a40fb55bca4a66bbee087244572cb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:07 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:33380
I20260806 16:11:07.235090 1425 tablet_service.cc:1514] Processing DeleteTablet for tablet 52a40fb55bca4a66bbee087244572cb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:07 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:57484
I20260806 16:11:07.236447 1908 tablet_replica.cc:330] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d: stopping tablet replica
I20260806 16:11:07.236454 1907 tablet_replica.cc:330] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: stopping tablet replica
I20260806 16:11:07.236820 1909 tablet_replica.cc:330] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286: stopping tablet replica
I20260806 16:11:07.236936 1907 raft_consensus.cc:2229] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:11:07.237459 1908 raft_consensus.cc:2229] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:11:07.237578 1907 raft_consensus.cc:2258] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:07.238122 1908 raft_consensus.cc:2258] T 52a40fb55bca4a66bbee087244572cb5 P 3555987774394799abaaf5403263b07d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:07.238157 1909 raft_consensus.cc:2229] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:11:07.238523 1909 raft_consensus.cc:2258] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:07.239506 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 1347
I20260806 16:11:07.244952 1909 ts_tablet_manager.cc:1905] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:07.248421 1909 ts_tablet_manager.cc:1918] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:07.248580 1909 log.cc:1196] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/52a40fb55bca4a66bbee087244572cb5
I20260806 16:11:07.248947 1909 ts_tablet_manager.cc:1939] T 52a40fb55bca4a66bbee087244572cb5 P d458b03f8c064c8a81e77c88aa271286: Deleting consensus metadata
I20260806 16:11:07.250670 1249 catalog_manager.cc:4841] TS d458b03f8c064c8a81e77c88aa271286 (127.8.49.3:39103): tablet 52a40fb55bca4a66bbee087244572cb5 (table test-table [id=e72b1fbda77741b094a4207d7474f222]) successfully deleted
I20260806 16:11:07.252252 1907 ts_tablet_manager.cc:1905] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:07.257717 1907 ts_tablet_manager.cc:1918] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:07.257923 1907 log.cc:1196] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/52a40fb55bca4a66bbee087244572cb5
I20260806 16:11:07.258402 1907 ts_tablet_manager.cc:1939] T 52a40fb55bca4a66bbee087244572cb5 P 81b904039b804de8ab176709db84351c: Deleting consensus metadata
I20260806 16:11:07.260713 1248 catalog_manager.cc:4841] TS 81b904039b804de8ab176709db84351c (127.8.49.2:32867): tablet 52a40fb55bca4a66bbee087244572cb5 (table test-table [id=e72b1fbda77741b094a4207d7474f222]) successfully deleted
W20260806 16:11:07.261464 1244 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:34655 (error 108)
I20260806 16:11:07.262686 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 1511
I20260806 16:11:07.263111 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.262642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:34655 (local address 127.0.0.1:57494)
0806 16:11:07.262881 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:07.262978 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:34655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 16:11:07.270018 1244 catalog_manager.cc:4584] TS 3555987774394799abaaf5403263b07d (127.8.49.1:34655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52a40fb55bca4a66bbee087244572cb5: Network error: Client connection negotiation failed: client connection to 127.8.49.1:34655: connect: Connection refused (error 111)
I20260806 16:11:07.297480 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 1687
I20260806 16:11:07.324568 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.324002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:34655 (local address 127.0.0.1:57508)
0806 16:11:07.324323 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:07.324437 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:34655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260806 16:11:07.325842 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 1225
2026-08-06T16:11:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (3187 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T16:11:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:07Z Disabled control of system clock
I20260806 16:11:07.376011 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:39443
--webserver_interface=127.8.49.62
--webserver_port=0
--builtin_ntp_servers=127.8.49.20:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:39443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:11:07.517704 1914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:07.518038 1914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:07.518078 1914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:07.522408 1914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:07.522506 1914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:07.522523 1914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:07.522540 1914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:07.522557 1914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:07.528017 1914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:39443
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:39443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:07.529418 1914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:07.531116 1914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:07.542161 1922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:07.542161 1924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:07.542192 1926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:07.542630 1914 server_base.cc:1034] running on GCE node
I20260806 16:11:07.543787 1914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:07.545608 1914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:07.547152 1914 hybrid_clock.cc:648] HybridClock initialized: now 1786032667547121 us; error 71 us; skew 500 ppm
I20260806 16:11:07.550401 1914 webserver.cc:460] Webserver started at http://127.8.49.62:34237/ using document root <none> and password file <none>
I20260806 16:11:07.551448 1914 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:07.551582 1914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:07.551923 1914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:07.555100 1914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "bfaacc8691ff4b72a21fdee0b6e129de"
format_stamp: "Formatted at 2026-08-06 16:11:07 on dist-test-slave-7z32"
server_key: "cc3ba332640720de7728320930ed21cc"
server_key_iv: "4d82533d3e52984c2db0196cf9fdee4f"
server_key_version: "encryptionkey@0"
I20260806 16:11:07.555963 1914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "bfaacc8691ff4b72a21fdee0b6e129de"
format_stamp: "Formatted at 2026-08-06 16:11:07 on dist-test-slave-7z32"
server_key: "cc3ba332640720de7728320930ed21cc"
server_key_iv: "4d82533d3e52984c2db0196cf9fdee4f"
server_key_version: "encryptionkey@0"
I20260806 16:11:07.562489 1914 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:11:07.566768 1932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:07.569031 1914 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:07.569334 1914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "bfaacc8691ff4b72a21fdee0b6e129de"
format_stamp: "Formatted at 2026-08-06 16:11:07 on dist-test-slave-7z32"
server_key: "cc3ba332640720de7728320930ed21cc"
server_key_iv: "4d82533d3e52984c2db0196cf9fdee4f"
server_key_version: "encryptionkey@0"
I20260806 16:11:07.569553 1914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:07.605084 1914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:07.606477 1914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:07.606804 1914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:07.619165 1914 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:39443
I20260806 16:11:07.619225 1996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:39443 every 8 connection(s)
I20260806 16:11:07.620673 1914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:07.626067 1997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:07.628835 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 1914
I20260806 16:11:07.629012 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:07.629392 8388 external_mini_cluster.cc:1428] Setting key e61189184e2d0af45d0218231ac70be6
I20260806 16:11:07.634810 1997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de: Bootstrap starting.
I20260806 16:11:07.637642 1997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:07.638779 1997 log.cc:824] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:07.642345 1997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de: No bootstrap required, opened a new log
I20260806 16:11:07.646386 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.631859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55458 (local address 127.8.49.62:39443)
0806 16:11:07.633095 (+ 1236us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:07.633120 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:07.633169 (+ 49us) server_negotiation.cc:407] Connection header received
0806 16:11:07.634009 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:07.634034 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:07.634539 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:07.635458 (+ 919us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:07.636546 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.637725 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.639350 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.640306 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.640496 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.641904 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:07.641911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:07.641934 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:07.643085 (+ 1151us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:07.643188 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:07.643505 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:07.643810 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:07.644115 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 16:11:07.644708 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":200,"threads_started":1}
I20260806 16:11:07.647312 1997 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaacc8691ff4b72a21fdee0b6e129de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39443 } }
I20260806 16:11:07.647706 1997 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:07.647802 1997 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfaacc8691ff4b72a21fdee0b6e129de, State: Initialized, Role: FOLLOWER
I20260806 16:11:07.648434 1997 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaacc8691ff4b72a21fdee0b6e129de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39443 } }
I20260806 16:11:07.648626 1997 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:07.648690 1997 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:07.648789 1997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:07.650103 1997 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaacc8691ff4b72a21fdee0b6e129de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39443 } }
I20260806 16:11:07.650550 1997 leader_election.cc:302] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfaacc8691ff4b72a21fdee0b6e129de; no voters:
I20260806 16:11:07.650961 1997 leader_election.cc:288] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:07.651234 2005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:07.651605 2005 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [term 1 LEADER]: Becoming Leader. State: Replica: bfaacc8691ff4b72a21fdee0b6e129de, State: Running, Role: LEADER
I20260806 16:11:07.652570 1997 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:07.652701 2005 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaacc8691ff4b72a21fdee0b6e129de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39443 } }
I20260806 16:11:07.656028 2007 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfaacc8691ff4b72a21fdee0b6e129de. Latest consensus state: current_term: 1 leader_uuid: "bfaacc8691ff4b72a21fdee0b6e129de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaacc8691ff4b72a21fdee0b6e129de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39443 } } }
I20260806 16:11:07.656054 2006 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfaacc8691ff4b72a21fdee0b6e129de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfaacc8691ff4b72a21fdee0b6e129de" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39443 } } }
I20260806 16:11:07.656292 2007 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:07.656296 2006 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de [sys.catalog]: This master's current role is: LEADER
W20260806 16:11:07.659842 2023 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:11:07.659978 2023 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:07.660094 2024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:07.661134 2024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:07.670550 2024 catalog_manager.cc:1347] Generated new cluster ID: 466788348e114e048122f7049ecd67dc
I20260806 16:11:07.670662 2024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:07.686692 2024 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:07.687908 2024 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:07.694855 2024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de: Generated new TSK 0
I20260806 16:11:07.696239 2024 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:07.706146 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39443
--builtin_ntp_servers=127.8.49.20:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:11:07.852330 2030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:07.852720 2030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:07.852768 2030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:07.858453 2030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:07.858577 2030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:07.858649 2030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:11:07.865100 2030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:07.866627 2030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:07.868408 2030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:07.881477 2040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:07.881531 2046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:07.881557 2043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:07.882141 2030 server_base.cc:1034] running on GCE node
I20260806 16:11:07.883073 2030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:07.884318 2030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:07.885632 2030 hybrid_clock.cc:648] HybridClock initialized: now 1786032667885571 us; error 50 us; skew 500 ppm
I20260806 16:11:07.889103 2030 webserver.cc:460] Webserver started at http://127.8.49.1:34361/ using document root <none> and password file <none>
I20260806 16:11:07.890022 2030 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:07.890125 2030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:07.890599 2030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:07.894757 2030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "2eab62089af24526ad668b1f1cd474e2"
format_stamp: "Formatted at 2026-08-06 16:11:07 on dist-test-slave-7z32"
server_key: "4971c4aeb1be8941e571b14fce2f605a"
server_key_iv: "65e897e4092dcba85f79177d09b2f275"
server_key_version: "encryptionkey@0"
I20260806 16:11:07.895787 2030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "2eab62089af24526ad668b1f1cd474e2"
format_stamp: "Formatted at 2026-08-06 16:11:07 on dist-test-slave-7z32"
server_key: "4971c4aeb1be8941e571b14fce2f605a"
server_key_iv: "65e897e4092dcba85f79177d09b2f275"
server_key_version: "encryptionkey@0"
I20260806 16:11:07.903275 2030 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260806 16:11:07.907907 2052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:07.911161 2030 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260806 16:11:07.911442 2030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "2eab62089af24526ad668b1f1cd474e2"
format_stamp: "Formatted at 2026-08-06 16:11:07 on dist-test-slave-7z32"
server_key: "4971c4aeb1be8941e571b14fce2f605a"
server_key_iv: "65e897e4092dcba85f79177d09b2f275"
server_key_version: "encryptionkey@0"
I20260806 16:11:07.911664 2030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:07.935539 2030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:07.936698 2030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:07.936995 2030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:07.938520 2030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:07.940157 2030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:07.940225 2030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:07.940286 2030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:07.940304 2030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:07.955673 2030 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:40803
I20260806 16:11:07.955732 2195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:40803 every 8 connection(s)
I20260806 16:11:07.957110 2030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:11:07.958428 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 2030
I20260806 16:11:07.958611 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:11:07.959041 8388 external_mini_cluster.cc:1428] Setting key 635bee849b94a36bcf5b9b65e4054a70
I20260806 16:11:07.962245 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39443
--builtin_ntp_servers=127.8.49.20:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:11:07.973476 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.960241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:55231 (local address 127.8.49.62:39443)
0806 16:11:07.960454 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:07.960461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:07.962772 (+ 2311us) server_negotiation.cc:407] Connection header received
0806 16:11:07.965363 (+ 2591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:07.965367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:07.965451 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:07.965567 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:07.966726 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.967744 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.969648 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.970163 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.970342 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.972456 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:07.972465 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:07.972470 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:07.972498 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:07.972541 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:07.972556 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:07.972701 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:07.973087 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 16:11:07.973200 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:11:07.974980 2198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:07.961066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:39443 (local address 127.8.49.1:55231)
0806 16:11:07.962513 (+ 1447us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:07.962619 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:07.965057 (+ 2438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:07.965799 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:07.965809 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:07.965917 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:07.966547 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.966562 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.967955 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.967959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.969397 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:07.969427 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:07.970482 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:07.970487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:07.970926 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:07.971959 (+ 1033us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:11:07.971961 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:07.972031 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:11:07.972160 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:07.972758 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:07.972848 (+ 90us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:07.972856 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 16:11:07.973043 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 16:11:07.973457 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":223,"threads_started":1}
I20260806 16:11:07.977423 2196 heartbeater.cc:347] Connected to a master server at 127.8.49.62:39443
I20260806 16:11:07.977910 2196 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:07.978989 2196 heartbeater.cc:511] Master 127.8.49.62:39443 requested a full tablet report, sending...
I20260806 16:11:07.981913 1954 ts_manager.cc:194] Registered new tserver with Master: 2eab62089af24526ad668b1f1cd474e2 (127.8.49.1:40803)
I20260806 16:11:07.983618 1954 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.1:55231
W20260806 16:11:08.119349 2199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:08.119823 2199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:08.119907 2199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:08.127420 2199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:08.127556 2199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:08.127704 2199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:11:08.135219 2199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:08.136955 2199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:08.139190 2199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:08.156010 2211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:08.156167 2206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:08.156010 2208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:08.156908 2199 server_base.cc:1034] running on GCE node
I20260806 16:11:08.158512 2199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:08.160538 2199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:08.161898 2199 hybrid_clock.cc:648] HybridClock initialized: now 1786032668161857 us; error 57 us; skew 500 ppm
I20260806 16:11:08.165444 2199 webserver.cc:460] Webserver started at http://127.8.49.2:46035/ using document root <none> and password file <none>
I20260806 16:11:08.166328 2199 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:08.166672 2199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:08.167188 2199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:08.171221 2199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "0a46d653ecdf45f0b2cfde5dbce802f2"
format_stamp: "Formatted at 2026-08-06 16:11:08 on dist-test-slave-7z32"
server_key: "5ea4471fa66d94310fb00127c88a109b"
server_key_iv: "add9a78433b8107696e665b34ebd872d"
server_key_version: "encryptionkey@0"
I20260806 16:11:08.172192 2199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "0a46d653ecdf45f0b2cfde5dbce802f2"
format_stamp: "Formatted at 2026-08-06 16:11:08 on dist-test-slave-7z32"
server_key: "5ea4471fa66d94310fb00127c88a109b"
server_key_iv: "add9a78433b8107696e665b34ebd872d"
server_key_version: "encryptionkey@0"
I20260806 16:11:08.178617 2199 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 16:11:08.183769 2217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:08.186731 2199 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:11:08.187077 2199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "0a46d653ecdf45f0b2cfde5dbce802f2"
format_stamp: "Formatted at 2026-08-06 16:11:08 on dist-test-slave-7z32"
server_key: "5ea4471fa66d94310fb00127c88a109b"
server_key_iv: "add9a78433b8107696e665b34ebd872d"
server_key_version: "encryptionkey@0"
I20260806 16:11:08.187319 2199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:08.207194 2199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:08.208323 2199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:08.208613 2199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:08.209513 2199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:08.211071 2199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:08.211162 2199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:08.211249 2199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:08.211297 2199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:08.226915 2199 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:35533
I20260806 16:11:08.226939 2365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:35533 every 8 connection(s)
I20260806 16:11:08.228307 2199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:11:08.236501 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 2199
I20260806 16:11:08.236922 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:11:08.237406 8388 external_mini_cluster.cc:1428] Setting key 748e6d358c47be1b259a2b0de2a03ab1
I20260806 16:11:08.241479 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39443
--builtin_ntp_servers=127.8.49.20:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:11:08.244071 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.231221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:57655 (local address 127.8.49.62:39443)
0806 16:11:08.231511 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:08.231516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:08.233315 (+ 1799us) server_negotiation.cc:407] Connection header received
0806 16:11:08.235303 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:08.235307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:08.235390 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:08.235523 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:08.236741 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.237529 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.239882 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.240374 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.240531 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.242742 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:08.242750 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:08.242756 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:08.242779 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:08.242813 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:08.242871 (+ 58us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:08.243147 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:08.243481 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 16:11:08.243596 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 16:11:08.245287 2369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.231842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:39443 (local address 127.8.49.2:57655)
0806 16:11:08.233058 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:08.233136 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:08.234976 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:08.235787 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:08.235796 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:08.235912 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:08.236546 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.236562 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.237718 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.237722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.239727 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.239741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.240691 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.240698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.241051 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.242337 (+ 1286us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:11:08.242340 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:08.242404 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:11:08.242512 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:08.243164 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:08.243176 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:08.243183 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 16:11:08.243350 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 16:11:08.243719 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":326,"threads_started":1}
I20260806 16:11:08.247795 2366 heartbeater.cc:347] Connected to a master server at 127.8.49.62:39443
I20260806 16:11:08.248216 2366 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:08.249159 2366 heartbeater.cc:511] Master 127.8.49.62:39443 requested a full tablet report, sending...
I20260806 16:11:08.251178 1954 ts_manager.cc:194] Registered new tserver with Master: 0a46d653ecdf45f0b2cfde5dbce802f2 (127.8.49.2:35533)
I20260806 16:11:08.252276 1954 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.2:57655
W20260806 16:11:08.384517 2370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:08.384931 2370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:08.384984 2370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:08.390620 2370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:08.390741 2370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:08.390813 2370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:11:08.396535 2370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:08.397835 2370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:08.399410 2370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:08.414007 2376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:08.414206 2375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:08.414484 2379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:08.415462 2370 server_base.cc:1034] running on GCE node
I20260806 16:11:08.416363 2370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:08.417274 2370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:08.418475 2370 hybrid_clock.cc:648] HybridClock initialized: now 1786032668418458 us; error 37 us; skew 500 ppm
I20260806 16:11:08.420975 2370 webserver.cc:460] Webserver started at http://127.8.49.3:36057/ using document root <none> and password file <none>
I20260806 16:11:08.421681 2370 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:08.421757 2370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:08.422014 2370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:08.424446 2370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c"
format_stamp: "Formatted at 2026-08-06 16:11:08 on dist-test-slave-7z32"
server_key: "915ce675aed0bc56046a6d07fb1a2bd0"
server_key_iv: "fdf266c7ff65da73afa60a7d7d0eb3f2"
server_key_version: "encryptionkey@0"
I20260806 16:11:08.425105 2370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c"
format_stamp: "Formatted at 2026-08-06 16:11:08 on dist-test-slave-7z32"
server_key: "915ce675aed0bc56046a6d07fb1a2bd0"
server_key_iv: "fdf266c7ff65da73afa60a7d7d0eb3f2"
server_key_version: "encryptionkey@0"
I20260806 16:11:08.430065 2370 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:11:08.433959 2385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:08.436384 2370 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:08.436646 2370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c"
format_stamp: "Formatted at 2026-08-06 16:11:08 on dist-test-slave-7z32"
server_key: "915ce675aed0bc56046a6d07fb1a2bd0"
server_key_iv: "fdf266c7ff65da73afa60a7d7d0eb3f2"
server_key_version: "encryptionkey@0"
I20260806 16:11:08.436827 2370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:08.463701 2370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:08.464558 2370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:08.464793 2370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:08.465487 2370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:08.466634 2370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:08.466718 2370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:08.466795 2370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:08.466867 2370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 16:11:08.479048 2370 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:36077
I20260806 16:11:08.479069 2536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:36077 every 8 connection(s)
I20260806 16:11:08.480087 2370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:11:08.486124 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 2370
I20260806 16:11:08.486358 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:11:08.486799 8388 external_mini_cluster.cc:1428] Setting key bb76cc5f84fa967c2e40472dd13001fa
I20260806 16:11:08.495005 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.482718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:44469 (local address 127.8.49.62:39443)
0806 16:11:08.483120 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:08.483125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:08.484464 (+ 1339us) server_negotiation.cc:407] Connection header received
0806 16:11:08.487139 (+ 2675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:08.487144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:08.487236 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:08.487399 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:08.489009 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.489688 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.491923 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.492235 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.492393 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.494246 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:08.494250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:08.494252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:08.494271 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:08.494298 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:08.494307 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:08.494390 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:08.494686 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 16:11:08.494760 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 16:11:08.496627 2539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.483211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:39443 (local address 127.8.49.3:44469)
0806 16:11:08.484203 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:08.484277 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:08.486499 (+ 2222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:08.487752 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:08.487763 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:08.487896 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:08.488761 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.488785 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.489889 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.489894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.491745 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.491761 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.492500 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.492504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.492769 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.493788 (+ 1019us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:11:08.493790 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:08.493855 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:11:08.493966 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:08.494454 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:08.494465 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:08.494473 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 16:11:08.494649 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 16:11:08.495355 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":135,"threads_started":1}
I20260806 16:11:08.498533 2537 heartbeater.cc:347] Connected to a master server at 127.8.49.62:39443
I20260806 16:11:08.498970 2537 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:08.499830 2537 heartbeater.cc:511] Master 127.8.49.62:39443 requested a full tablet report, sending...
I20260806 16:11:08.501639 1954 ts_manager.cc:194] Registered new tserver with Master: 1dcd1c0594c9414fa1d44ebec1e1ac0c (127.8.49.3:36077)
I20260806 16:11:08.502548 1954 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.3:44469
I20260806 16:11:08.503438 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:11:08.512810 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.505966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.8.49.62:39443)
0806 16:11:08.506201 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:08.506209 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:08.506327 (+ 118us) server_negotiation.cc:407] Connection header received
0806 16:11:08.506389 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:08.506391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:08.506439 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:08.506536 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:08.507141 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.508443 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.510156 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.510610 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.510761 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.512134 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:08.512139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:08.512142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:08.512170 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:08.512193 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:08.512204 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:08.512366 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:08.512531 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 16:11:08.512592 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 16:11:08.516956 1953 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:11:08.520787 1953 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:11:08.541463 2550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.533367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:36077 (local address 127.0.0.1:40052)
0806 16:11:08.534322 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:08.534391 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:08.534552 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:08.535681 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:08.535685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:08.535708 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:08.535854 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.535865 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.537374 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.537380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.538899 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.538913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.540658 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.540664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.540817 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.540941 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 16:11:08.541066 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 16:11:08.541260 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"mutex_wait_us":84,"spinlock_wait_cycles":11136,"thread_start_us":138,"threads_started":1}
I20260806 16:11:08.541674 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.533467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54324 (local address 127.8.49.2:35533)
0806 16:11:08.534080 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:08.534096 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:08.534709 (+ 613us) server_negotiation.cc:407] Connection header received
0806 16:11:08.534806 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:08.534811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:08.535095 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:08.535263 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:08.535894 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.537652 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.539264 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.540165 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.540290 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.540539 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:08.541350 (+ 811us) server_negotiation.cc:299] Negotiation successful
0806 16:11:08.541481 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":198,"threads_started":1}
I20260806 16:11:08.541563 2555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.534364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40803 (local address 127.0.0.1:38104)
0806 16:11:08.535060 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:08.535091 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:08.535229 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:08.535921 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:08.535925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:08.535948 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:08.536060 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.536067 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.537893 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.537898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.539067 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.539083 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.540935 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.540939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.541091 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.541124 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 16:11:08.541221 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 16:11:08.541340 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":144,"thread_start_us":140,"threads_started":1}
I20260806 16:11:08.542155 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.533327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:35533 (local address 127.0.0.1:54324)
0806 16:11:08.534322 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:08.534391 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:08.534552 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:08.535398 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:08.535405 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:08.535429 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:08.535686 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.535697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.537844 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.537848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.539122 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.539130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.540608 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.540615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.540752 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.540941 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 16:11:08.541066 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 16:11:08.541949 (+ 883us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":825,"spinlock_wait_cycles":8704,"thread_start_us":114,"threads_started":1}
I20260806 16:11:08.542443 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.534498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40052 (local address 127.8.49.3:36077)
0806 16:11:08.534997 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:08.535015 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:08.535051 (+ 36us) server_negotiation.cc:407] Connection header received
0806 16:11:08.535177 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:08.535185 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:08.535464 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:08.535679 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:08.536084 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.537235 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.539065 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.540391 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.540903 (+ 512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.541115 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:08.541196 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 16:11:08.541358 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":170,"threads_started":1}
I20260806 16:11:08.542738 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.534997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38104 (local address 127.8.49.1:40803)
0806 16:11:08.535483 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:08.535489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:08.535508 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:11:08.535576 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:08.535605 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:08.535786 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:08.535940 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:08.536258 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.537740 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.539330 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.540408 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.540571 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.540859 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:08.542371 (+ 1512us) server_negotiation.cc:299] Negotiation successful
0806 16:11:08.542542 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":172,"threads_started":1}
I20260806 16:11:08.546188 2278 tablet_service.cc:1467] Processing CreateTablet for tablet 62ad21834db44b9fbd474721c25e78f1 (DEFAULT_TABLE table=test-table [id=b8d9cb50379b48ecbbb2cd0e23a23dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:08.546188 2107 tablet_service.cc:1467] Processing CreateTablet for tablet 62ad21834db44b9fbd474721c25e78f1 (DEFAULT_TABLE table=test-table [id=b8d9cb50379b48ecbbb2cd0e23a23dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:08.546284 2449 tablet_service.cc:1467] Processing CreateTablet for tablet 62ad21834db44b9fbd474721c25e78f1 (DEFAULT_TABLE table=test-table [id=b8d9cb50379b48ecbbb2cd0e23a23dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:08.547616 2107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ad21834db44b9fbd474721c25e78f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:08.547616 2278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ad21834db44b9fbd474721c25e78f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:08.547915 2449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ad21834db44b9fbd474721c25e78f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:08.556958 2561 tablet_bootstrap.cc:492] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2: Bootstrap starting.
I20260806 16:11:08.558109 2564 tablet_bootstrap.cc:492] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: Bootstrap starting.
I20260806 16:11:08.558130 2563 tablet_bootstrap.cc:492] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: Bootstrap starting.
I20260806 16:11:08.560969 2563 tablet_bootstrap.cc:654] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:08.561014 2561 tablet_bootstrap.cc:654] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:08.561014 2564 tablet_bootstrap.cc:654] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:08.561882 2563 log.cc:824] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:08.562135 2564 log.cc:824] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:08.562144 2561 log.cc:824] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:08.564870 2561 tablet_bootstrap.cc:492] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2: No bootstrap required, opened a new log
I20260806 16:11:08.564945 2563 tablet_bootstrap.cc:492] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: No bootstrap required, opened a new log
I20260806 16:11:08.564946 2564 tablet_bootstrap.cc:492] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: No bootstrap required, opened a new log
I20260806 16:11:08.565187 2564 ts_tablet_manager.cc:1397] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:11:08.565187 2563 ts_tablet_manager.cc:1397] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 16:11:08.565203 2561 ts_tablet_manager.cc:1397] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260806 16:11:08.570734 2563 raft_consensus.cc:348] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 35533 } } peers { permanent_uuid: "2eab62089af24526ad668b1f1cd474e2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40803 } } peers { permanent_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 36077 } }
I20260806 16:11:08.571256 2563 raft_consensus.cc:374] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:08.570922 2561 raft_consensus.cc:348] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 35533 } } peers { permanent_uuid: "2eab62089af24526ad668b1f1cd474e2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40803 } } peers { permanent_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 36077 } }
I20260806 16:11:08.571318 2563 raft_consensus.cc:729] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a46d653ecdf45f0b2cfde5dbce802f2, State: Initialized, Role: FOLLOWER
I20260806 16:11:08.571360 2561 raft_consensus.cc:374] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:08.571455 2561 raft_consensus.cc:729] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eab62089af24526ad668b1f1cd474e2, State: Initialized, Role: FOLLOWER
I20260806 16:11:08.571239 2564 raft_consensus.cc:348] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 35533 } } peers { permanent_uuid: "2eab62089af24526ad668b1f1cd474e2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40803 } } peers { permanent_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 36077 } }
I20260806 16:11:08.571566 2564 raft_consensus.cc:374] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:08.571625 2564 raft_consensus.cc:729] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcd1c0594c9414fa1d44ebec1e1ac0c, State: Initialized, Role: FOLLOWER
I20260806 16:11:08.571928 2563 consensus_queue.cc:260] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 35533 } } peers { permanent_uuid: "2eab62089af24526ad668b1f1cd474e2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40803 } } peers { permanent_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 36077 } }
I20260806 16:11:08.572273 2564 consensus_queue.cc:260] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 35533 } } peers { permanent_uuid: "2eab62089af24526ad668b1f1cd474e2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40803 } } peers { permanent_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 36077 } }
I20260806 16:11:08.572451 2561 consensus_queue.cc:260] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 35533 } } peers { permanent_uuid: "2eab62089af24526ad668b1f1cd474e2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40803 } } peers { permanent_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 36077 } }
I20260806 16:11:08.573104 2366 heartbeater.cc:503] Master 127.8.49.62:39443 was elected leader, sending a full tablet report...
I20260806 16:11:08.573376 2564 ts_tablet_manager.cc:1428] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260806 16:11:08.573423 2196 heartbeater.cc:503] Master 127.8.49.62:39443 was elected leader, sending a full tablet report...
I20260806 16:11:08.573781 2537 heartbeater.cc:503] Master 127.8.49.62:39443 was elected leader, sending a full tablet report...
I20260806 16:11:08.573856 2561 ts_tablet_manager.cc:1428] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20260806 16:11:08.575130 2563 ts_tablet_manager.cc:1428] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260806 16:11:08.650853 2569 raft_consensus.cc:482] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:11:08.651657 2569 raft_consensus.cc:504] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 35533 } } peers { permanent_uuid: "2eab62089af24526ad668b1f1cd474e2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40803 } } peers { permanent_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 36077 } }
I20260806 16:11:08.653735 2569 leader_election.cc:288] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2eab62089af24526ad668b1f1cd474e2 (127.8.49.1:40803), 1dcd1c0594c9414fa1d44ebec1e1ac0c (127.8.49.3:36077)
I20260806 16:11:08.660290 2369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.654089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40803 (local address 127.8.49.2:59263)
0806 16:11:08.654246 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:08.654267 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:08.654411 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:08.654970 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:08.654974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:08.654999 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:08.655202 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.655213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.656331 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.656334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.657793 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.657809 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.659324 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.659330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.659674 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.659832 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 16:11:08.659920 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 16:11:08.660058 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":200}
I20260806 16:11:08.660522 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.654596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:46785 (local address 127.8.49.3:36077)
0806 16:11:08.654741 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:08.654745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:08.655337 (+ 592us) server_negotiation.cc:407] Connection header received
0806 16:11:08.655419 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:08.655423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:08.655497 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:08.655611 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:08.655970 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.656547 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.658197 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.659081 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.659234 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.659322 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:08.660136 (+ 814us) server_negotiation.cc:299] Negotiation successful
0806 16:11:08.660247 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 16:11:08.660517 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.654400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:59263 (local address 127.8.49.1:40803)
0806 16:11:08.654596 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:08.654604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:08.654630 (+ 26us) server_negotiation.cc:407] Connection header received
0806 16:11:08.654715 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:08.654719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:08.654781 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:08.654945 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:08.655383 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.656118 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.658197 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.659061 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.659233 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.659322 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:08.660134 (+ 812us) server_negotiation.cc:299] Negotiation successful
0806 16:11:08.660247 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 16:11:08.660334 2572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.654459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:36077 (local address 127.8.49.2:46785)
0806 16:11:08.655092 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:08.655108 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:08.655242 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:08.655678 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:08.655681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:08.655705 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:08.655826 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.655832 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.656691 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.656694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.658008 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:08.658016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.659324 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.659330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:08.659706 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.659758 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 16:11:08.659897 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 16:11:08.660051 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":71,"thread_start_us":165,"threads_started":1}
I20260806 16:11:08.661805 2479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ad21834db44b9fbd474721c25e78f1" candidate_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" is_pre_election: true
I20260806 16:11:08.661926 2129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ad21834db44b9fbd474721c25e78f1" candidate_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eab62089af24526ad668b1f1cd474e2" is_pre_election: true
I20260806 16:11:08.662263 2479 raft_consensus.cc:2454] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a46d653ecdf45f0b2cfde5dbce802f2 in term 0.
I20260806 16:11:08.662467 2129 raft_consensus.cc:2454] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a46d653ecdf45f0b2cfde5dbce802f2 in term 0.
I20260806 16:11:08.663151 2222 leader_election.cc:302] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a46d653ecdf45f0b2cfde5dbce802f2, 1dcd1c0594c9414fa1d44ebec1e1ac0c; no voters:
I20260806 16:11:08.663612 2569 raft_consensus.cc:2781] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:11:08.663726 2569 raft_consensus.cc:482] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:11:08.663775 2569 raft_consensus.cc:3037] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:08.665681 2569 raft_consensus.cc:504] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 35533 } } peers { permanent_uuid: "2eab62089af24526ad668b1f1cd474e2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40803 } } peers { permanent_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 36077 } }
I20260806 16:11:08.666298 2569 leader_election.cc:288] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [CANDIDATE]: Term 1 election: Requested vote from peers 2eab62089af24526ad668b1f1cd474e2 (127.8.49.1:40803), 1dcd1c0594c9414fa1d44ebec1e1ac0c (127.8.49.3:36077)
I20260806 16:11:08.666913 2129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ad21834db44b9fbd474721c25e78f1" candidate_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eab62089af24526ad668b1f1cd474e2"
I20260806 16:11:08.667111 2129 raft_consensus.cc:3037] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:08.667482 2479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ad21834db44b9fbd474721c25e78f1" candidate_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c"
I20260806 16:11:08.667810 2479 raft_consensus.cc:3037] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:08.668766 2129 raft_consensus.cc:2454] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a46d653ecdf45f0b2cfde5dbce802f2 in term 1.
I20260806 16:11:08.669308 2225 leader_election.cc:302] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a46d653ecdf45f0b2cfde5dbce802f2, 2eab62089af24526ad668b1f1cd474e2; no voters:
I20260806 16:11:08.669597 2569 raft_consensus.cc:2781] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:08.669911 2569 raft_consensus.cc:686] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 1 LEADER]: Becoming Leader. State: Replica: 0a46d653ecdf45f0b2cfde5dbce802f2, State: Running, Role: LEADER
I20260806 16:11:08.670365 2569 consensus_queue.cc:237] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 35533 } } peers { permanent_uuid: "2eab62089af24526ad668b1f1cd474e2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40803 } } peers { permanent_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 36077 } }
I20260806 16:11:08.671325 2479 raft_consensus.cc:2454] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a46d653ecdf45f0b2cfde5dbce802f2 in term 1.
I20260806 16:11:08.675508 1954 catalog_manager.cc:5495] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a46d653ecdf45f0b2cfde5dbce802f2 (127.8.49.2). New cstate: current_term: 1 leader_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a46d653ecdf45f0b2cfde5dbce802f2" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 35533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2eab62089af24526ad668b1f1cd474e2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 36077 } health_report { overall_health: UNKNOWN } } }
W20260806 16:11:08.709106 2197 tablet.cc:2369] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:11:08.715305 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.709687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50530 (local address 127.8.49.2:35533)
0806 16:11:08.709867 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:08.709871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:08.709892 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:11:08.710106 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:08.710109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:08.710196 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:08.710669 (+ 473us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:08.711291 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.711854 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.712852 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:08.713097 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:08.713553 (+ 456us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:08.714082 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:08.714571 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:08.714750 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:08.714909 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 16:11:08.714958 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 16:11:08.726064 2129 raft_consensus.cc:1261] T 62ad21834db44b9fbd474721c25e78f1 P 2eab62089af24526ad668b1f1cd474e2 [term 1 FOLLOWER]: Refusing update from remote peer 0a46d653ecdf45f0b2cfde5dbce802f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:08.727053 2479 raft_consensus.cc:1261] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c [term 1 FOLLOWER]: Refusing update from remote peer 0a46d653ecdf45f0b2cfde5dbce802f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:08.727236 2573 consensus_queue.cc:1035] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2eab62089af24526ad668b1f1cd474e2" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:08.728439 2573 consensus_queue.cc:1035] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcd1c0594c9414fa1d44ebec1e1ac0c" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 36077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 16:11:08.729910 2368 tablet.cc:2369] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:11:08.731336 2538 tablet.cc:2369] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:11:08.740893 2577 mvcc.cc:205] Tried to move back new op lower bound from 7315589811090579456 to 7315589810877489152. Current Snapshot: MvccSnapshot[applied={T|T < 7315589811090579456}]
I20260806 16:11:08.740885 2580 mvcc.cc:205] Tried to move back new op lower bound from 7315589811090579456 to 7315589810877489152. Current Snapshot: MvccSnapshot[applied={T|T < 7315589811090579456}]
I20260806 16:11:08.752389 1953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:08.752879 1953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:08.759130 1953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bfaacc8691ff4b72a21fdee0b6e129de: Sending DeleteTablet for 3 replicas of tablet 62ad21834db44b9fbd474721c25e78f1
I20260806 16:11:08.761395 2107 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ad21834db44b9fbd474721c25e78f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:08 UTC) from {username='slave'} at 127.0.0.1:38104
I20260806 16:11:08.761404 2449 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ad21834db44b9fbd474721c25e78f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:08 UTC) from {username='slave'} at 127.0.0.1:40052
I20260806 16:11:08.762247 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 2030
I20260806 16:11:08.763463 2278 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ad21834db44b9fbd474721c25e78f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:08 UTC) from {username='slave'} at 127.0.0.1:54324
I20260806 16:11:08.766461 2590 tablet_replica.cc:330] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: stopping tablet replica
I20260806 16:11:08.767246 2590 raft_consensus.cc:2229] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:11:08.768043 2590 raft_consensus.cc:2258] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:08.773097 2590 ts_tablet_manager.cc:1905] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:08.774756 2588 tablet_replica.cc:330] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: stopping tablet replica
I20260806 16:11:08.775651 2588 raft_consensus.cc:2229] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:11:08.776261 2588 raft_consensus.cc:2258] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:08.778393 2590 ts_tablet_manager.cc:1918] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:08.778633 2590 log.cc:1196] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/62ad21834db44b9fbd474721c25e78f1
I20260806 16:11:08.780095 2590 ts_tablet_manager.cc:1939] T 62ad21834db44b9fbd474721c25e78f1 P 0a46d653ecdf45f0b2cfde5dbce802f2: Deleting consensus metadata
I20260806 16:11:08.781234 2588 ts_tablet_manager.cc:1905] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:08.782603 1940 catalog_manager.cc:4841] TS 0a46d653ecdf45f0b2cfde5dbce802f2 (127.8.49.2:35533): tablet 62ad21834db44b9fbd474721c25e78f1 (table test-table [id=b8d9cb50379b48ecbbb2cd0e23a23dd2]) successfully deleted
I20260806 16:11:08.786152 2588 ts_tablet_manager.cc:1918] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:08.786375 2588 log.cc:1196] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/62ad21834db44b9fbd474721c25e78f1
I20260806 16:11:08.786787 2588 ts_tablet_manager.cc:1939] T 62ad21834db44b9fbd474721c25e78f1 P 1dcd1c0594c9414fa1d44ebec1e1ac0c: Deleting consensus metadata
W20260806 16:11:08.787844 1940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:40803 (error 108)
I20260806 16:11:08.789072 1936 catalog_manager.cc:4841] TS 1dcd1c0594c9414fa1d44ebec1e1ac0c (127.8.49.3:36077): tablet 62ad21834db44b9fbd474721c25e78f1 (table test-table [id=b8d9cb50379b48ecbbb2cd0e23a23dd2]) successfully deleted
I20260806 16:11:08.789531 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.789078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40803 (local address 127.0.0.1:36114)
0806 16:11:08.789324 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:08.789412 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260806 16:11:08.789901 1940 catalog_manager.cc:4584] TS 2eab62089af24526ad668b1f1cd474e2 (127.8.49.1:40803): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62ad21834db44b9fbd474721c25e78f1: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40803: connect: Connection refused (error 111)
I20260806 16:11:08.790081 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 2199
I20260806 16:11:08.829718 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 2370
I20260806 16:11:08.844497 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.843644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40803 (local address 127.0.0.1:36116)
0806 16:11:08.843995 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:08.844303 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 16:11:08.846136 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:08.845655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:40803 (local address 127.0.0.1:36124)
0806 16:11:08.845889 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:08.846006 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:40803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 16:11:08.850713 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 1914
2026-08-06T16:11:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1525 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:08 dist-test-slave-7z32 krb5kdc[2597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:08 dist-test-slave-7z32 krb5kdc[2597](info): set up 2 sockets
Aug 06 16:11:08 dist-test-slave-7z32 krb5kdc[2597](info): commencing operation
krb5kdc: starting...
W20260806 16:11:10.247937 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:10 dist-test-slave-7z32 krb5kdc[2597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:11:10.440531 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:34937
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:32931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:34937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:10.612778 2614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:10.613227 2614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:10.613277 2614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:10.618112 2614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:10.618242 2614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:10.618264 2614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:10.618285 2614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:10.618302 2614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:10.627080 2614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:32931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:34937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:34937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:10.628892 2614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:10.630445 2614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:10.644718 2621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:10.644734 2620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:10.644718 2623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:10.645226 2614 server_base.cc:1034] running on GCE node
I20260806 16:11:10.646900 2614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:10.648736 2614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:10.652478 2614 hybrid_clock.cc:648] HybridClock initialized: now 1786032670652264 us; error 210 us; skew 500 ppm
Aug 06 16:11:10 dist-test-slave-7z32 krb5kdc[2597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032670, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:10.657814 2614 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:11:10.659595 2614 webserver.cc:460] Webserver started at http://127.8.49.62:46781/ using document root <none> and password file <none>
I20260806 16:11:10.660445 2614 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:10.660564 2614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:10.660883 2614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:10.663942 2614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "d244413bc3094203922825f3dbb6c6c5"
format_stamp: "Formatted at 2026-08-06 16:11:10 on dist-test-slave-7z32"
server_key: "4a7301d51dda36f7d219b422aba666b0"
server_key_iv: "97684131c43559245f9364b2bc0aed0e"
server_key_version: "encryptionkey@0"
I20260806 16:11:10.664736 2614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "d244413bc3094203922825f3dbb6c6c5"
format_stamp: "Formatted at 2026-08-06 16:11:10 on dist-test-slave-7z32"
server_key: "4a7301d51dda36f7d219b422aba666b0"
server_key_iv: "97684131c43559245f9364b2bc0aed0e"
server_key_version: "encryptionkey@0"
I20260806 16:11:10.670809 2614 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 16:11:10.675191 2633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:10.677539 2614 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:11:10.677841 2614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "d244413bc3094203922825f3dbb6c6c5"
format_stamp: "Formatted at 2026-08-06 16:11:10 on dist-test-slave-7z32"
server_key: "4a7301d51dda36f7d219b422aba666b0"
server_key_iv: "97684131c43559245f9364b2bc0aed0e"
server_key_version: "encryptionkey@0"
I20260806 16:11:10.678061 2614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:10.710024 2614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:10.718240 2614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:10.718606 2614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:10.731282 2614 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:34937
I20260806 16:11:10.731475 2694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:34937 every 8 connection(s)
I20260806 16:11:10.732849 2614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:10.735309 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 2614
I20260806 16:11:10.735504 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:10.736044 8388 external_mini_cluster.cc:1428] Setting key 60592bff37f01cddf8339e08818c4c9a
I20260806 16:11:10.737124 2695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:10.746412 2695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5: Bootstrap starting.
I20260806 16:11:10.750742 2695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5: Neither blocks nor log segments found. Creating new log.
Aug 06 16:11:10 dist-test-slave-7z32 krb5kdc[2597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:10.752395 2695 log.cc:824] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:10.757866 2695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5: No bootstrap required, opened a new log
I20260806 16:11:10.762455 2699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:10.738054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40344 (local address 127.8.49.62:34937)
0806 16:11:10.738729 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:10.738751 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:10.738806 (+ 55us) server_negotiation.cc:407] Connection header received
0806 16:11:10.739788 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:10.739814 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:10.740296 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:10.740938 (+ 642us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:10.742105 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:10.744015 (+ 1910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:10.745796 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:10.747719 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:10.748157 (+ 438us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:10.752757 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:10.752767 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:10.752796 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:10.752855 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:10.756190 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:10.757519 (+ 1329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:10.757526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:10.757534 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:10.757701 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:10.759249 (+ 1548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:10.759253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:10.759255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:10.759987 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:10.760190 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:10.760818 (+ 628us) server_negotiation.cc:299] Negotiation successful
0806 16:11:10.761199 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":235,"threads_started":1}
I20260806 16:11:10.765913 2695 raft_consensus.cc:348] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d244413bc3094203922825f3dbb6c6c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34937 } }
I20260806 16:11:10.766605 2695 raft_consensus.cc:374] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:10.766718 2695 raft_consensus.cc:729] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d244413bc3094203922825f3dbb6c6c5, State: Initialized, Role: FOLLOWER
I20260806 16:11:10.767458 2695 consensus_queue.cc:260] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d244413bc3094203922825f3dbb6c6c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34937 } }
I20260806 16:11:10.767652 2695 raft_consensus.cc:388] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:10.767735 2695 raft_consensus.cc:482] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:10.767867 2695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:10.769305 2695 raft_consensus.cc:504] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d244413bc3094203922825f3dbb6c6c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34937 } }
I20260806 16:11:10.769825 2695 leader_election.cc:302] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d244413bc3094203922825f3dbb6c6c5; no voters:
I20260806 16:11:10.770298 2695 leader_election.cc:288] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:10.770519 2701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:10.771353 2701 raft_consensus.cc:686] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [term 1 LEADER]: Becoming Leader. State: Replica: d244413bc3094203922825f3dbb6c6c5, State: Running, Role: LEADER
I20260806 16:11:10.771903 2701 consensus_queue.cc:237] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d244413bc3094203922825f3dbb6c6c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34937 } }
I20260806 16:11:10.772502 2695 sys_catalog.cc:564] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:10.774173 2702 sys_catalog.cc:455] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d244413bc3094203922825f3dbb6c6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d244413bc3094203922825f3dbb6c6c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34937 } } }
I20260806 16:11:10.774417 2702 sys_catalog.cc:458] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:10.775251 2703 sys_catalog.cc:455] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d244413bc3094203922825f3dbb6c6c5. Latest consensus state: current_term: 1 leader_uuid: "d244413bc3094203922825f3dbb6c6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d244413bc3094203922825f3dbb6c6c5" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 34937 } } }
I20260806 16:11:10.775439 2703 sys_catalog.cc:458] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5 [sys.catalog]: This master's current role is: LEADER
W20260806 16:11:10.781257 2719 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:11:10.781389 2719 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:10.781510 2710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:10.782392 2710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:10.791574 2710 catalog_manager.cc:1347] Generated new cluster ID: 531dafb4da724f11b08afeef5c7651d0
I20260806 16:11:10.791760 2710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:10.802654 2710 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:10.805191 2710 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:10.818780 2710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5: Generated new TSK 0
I20260806 16:11:10.820734 2710 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.49.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
Entry for principal kudu/127.8.49.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab.
I20260806 16:11:10.884248 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34937
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:32931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:11.050138 2728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:11.050557 2728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:11.050647 2728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:11.055579 2728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:11.055719 2728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:11.055835 2728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:11:11.062117 2728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:32931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.1.keytab
--principal=kudu/127.8.49.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:11.063556 2728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:11.064776 2728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:11.077694 2734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:11.078045 2728 server_base.cc:1034] running on GCE node
W20260806 16:11:11.078361 2733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:11.078281 2736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:11.079176 2728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:11.080209 2728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:11.081620 2728 hybrid_clock.cc:648] HybridClock initialized: now 1786032671081550 us; error 77 us; skew 500 ppm
Aug 06 16:11:11 dist-test-slave-7z32 krb5kdc[2597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032671, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:11.087007 2728 init.cc:377] Logged in from keytab as kudu/127.8.49.1@KRBTEST.COM (short username kudu)
I20260806 16:11:11.088800 2728 webserver.cc:460] Webserver started at http://127.8.49.1:42033/ using document root <none> and password file <none>
I20260806 16:11:11.090059 2728 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:11.090171 2728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:11.090463 2728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:11.094403 2728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "857c9e28469d47488446bfe618c0a489"
format_stamp: "Formatted at 2026-08-06 16:11:11 on dist-test-slave-7z32"
server_key: "46bc9862143cdb6cca78f427a40bd8ad"
server_key_iv: "7889fad9a469e780e0f11d74daa9a1d5"
server_key_version: "encryptionkey@0"
I20260806 16:11:11.095391 2728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "857c9e28469d47488446bfe618c0a489"
format_stamp: "Formatted at 2026-08-06 16:11:11 on dist-test-slave-7z32"
server_key: "46bc9862143cdb6cca78f427a40bd8ad"
server_key_iv: "7889fad9a469e780e0f11d74daa9a1d5"
server_key_version: "encryptionkey@0"
I20260806 16:11:11.101179 2728 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:11:11.105409 2744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:11.107836 2728 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:11.108070 2728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "857c9e28469d47488446bfe618c0a489"
format_stamp: "Formatted at 2026-08-06 16:11:11 on dist-test-slave-7z32"
server_key: "46bc9862143cdb6cca78f427a40bd8ad"
server_key_iv: "7889fad9a469e780e0f11d74daa9a1d5"
server_key_version: "encryptionkey@0"
I20260806 16:11:11.108255 2728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:11.140189 2728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:11.145203 2728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:11.145445 2728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:11.146435 2728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:11.147688 2728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:11.147775 2728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:11.147859 2728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:11.147904 2728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:11.162312 2728 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:35517
I20260806 16:11:11.162335 2892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:35517 every 8 connection(s)
I20260806 16:11:11.163862 2728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:11:11.169368 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 2728
I20260806 16:11:11.169602 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:11:11.170022 8388 external_mini_cluster.cc:1428] Setting key 6c96b2483e16f146e052de0d8e21f287
Aug 06 16:11:11 dist-test-slave-7z32 krb5kdc[2597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032671, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.1@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.49.2@KRBTEST.COM; defaulting to no policy
I20260806 16:11:11.188547 2699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.166790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:35639 (local address 127.8.49.62:34937)
0806 16:11:11.167053 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:11.167058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:11.168298 (+ 1240us) server_negotiation.cc:407] Connection header received
0806 16:11:11.170320 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:11.170326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:11.170421 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:11.170590 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:11.172817 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.173901 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.176750 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.177203 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.177380 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.182265 (+ 4885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:11.182270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:11.182273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:11.182323 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:11.184806 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:11.185999 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:11.186004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:11.186006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:11.186138 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:11.186948 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:11.186952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:11.186953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:11.187247 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:11.187366 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:11.188139 (+ 773us) server_negotiation.cc:299] Negotiation successful
0806 16:11:11.188361 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.2@KRBTEST.COM" created.
I20260806 16:11:11.189934 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.167371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:34937 (local address 127.8.49.1:35639)
0806 16:11:11.168075 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:11.168139 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:11.169949 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:11.170986 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:11.171003 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:11.171620 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:11.172522 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.172550 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.174253 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.174261 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.176484 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.176513 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.177491 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.177496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.177798 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.178975 (+ 1177us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:11.179003 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:11.181757 (+ 2754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:11.185061 (+ 3304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:11.185078 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:11.185112 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:11.185742 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:11.186327 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:11.186331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:11.186332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:11.186623 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:11.187411 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:11.187419 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:11.187779 (+ 360us) client_negotiation.cc:769] Sending connection context
0806 16:11:11.187996 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 16:11:11.188467 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":187,"threads_started":1}
I20260806 16:11:11.191745 2893 heartbeater.cc:347] Connected to a master server at 127.8.49.62:34937
I20260806 16:11:11.192102 2893 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:11.192772 2893 heartbeater.cc:511] Master 127.8.49.62:34937 requested a full tablet report, sending...
I20260806 16:11:11.194697 2653 ts_manager.cc:194] Registered new tserver with Master: 857c9e28469d47488446bfe618c0a489 (127.8.49.1:35517)
I20260806 16:11:11.196318 2653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.1@KRBTEST.COM'} at 127.8.49.1:35639
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
Entry for principal kudu/127.8.49.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab.
I20260806 16:11:11.220803 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34937
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:32931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:11.382560 2900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:11.383062 2900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:11.383129 2900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:11.388310 2900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:11.388438 2900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:11.388515 2900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:11:11.396790 2900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:32931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.2.keytab
--principal=kudu/127.8.49.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:11.398430 2900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:11.399889 2900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:11.415160 2910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:11.415226 2915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:11.415160 2908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:11.415727 2900 server_base.cc:1034] running on GCE node
I20260806 16:11:11.416536 2900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:11.417647 2900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:11.418982 2900 hybrid_clock.cc:648] HybridClock initialized: now 1786032671418943 us; error 68 us; skew 500 ppm
Aug 06 16:11:11 dist-test-slave-7z32 krb5kdc[2597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032671, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:11.430754 2900 init.cc:377] Logged in from keytab as kudu/127.8.49.2@KRBTEST.COM (short username kudu)
I20260806 16:11:11.432441 2900 webserver.cc:460] Webserver started at http://127.8.49.2:42637/ using document root <none> and password file <none>
I20260806 16:11:11.433313 2900 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:11.433413 2900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:11.433738 2900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:11.437547 2900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "907af58cb6e64be68c6784a397371865"
format_stamp: "Formatted at 2026-08-06 16:11:11 on dist-test-slave-7z32"
server_key: "d02f84af58802d46ffaf89f3515a35be"
server_key_iv: "bfbd40d1c177e86c16a108342db20142"
server_key_version: "encryptionkey@0"
I20260806 16:11:11.438526 2900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "907af58cb6e64be68c6784a397371865"
format_stamp: "Formatted at 2026-08-06 16:11:11 on dist-test-slave-7z32"
server_key: "d02f84af58802d46ffaf89f3515a35be"
server_key_iv: "bfbd40d1c177e86c16a108342db20142"
server_key_version: "encryptionkey@0"
I20260806 16:11:11.444876 2900 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260806 16:11:11.448984 2925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:11.451501 2900 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:11:11.451699 2900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "907af58cb6e64be68c6784a397371865"
format_stamp: "Formatted at 2026-08-06 16:11:11 on dist-test-slave-7z32"
server_key: "d02f84af58802d46ffaf89f3515a35be"
server_key_iv: "bfbd40d1c177e86c16a108342db20142"
server_key_version: "encryptionkey@0"
I20260806 16:11:11.451933 2900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:11.470052 2900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:11.476293 2900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:11.476532 2900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:11.477382 2900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:11.478638 2900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:11.478942 2900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 16:11:11.479065 2900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:11.479122 2900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:11.494099 2900 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:36937
I20260806 16:11:11.494273 3068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:36937 every 8 connection(s)
I20260806 16:11:11.495747 2900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:11:11.504741 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 2900
I20260806 16:11:11.504945 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:11:11.505535 8388 external_mini_cluster.cc:1428] Setting key fa05ae8572aa076cd585a3d97b701f94
Aug 06 16:11:11 dist-test-slave-7z32 krb5kdc[2597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032671, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.2@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:11.518958 2699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.498539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:36999 (local address 127.8.49.62:34937)
0806 16:11:11.498968 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:11.498973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:11.500358 (+ 1385us) server_negotiation.cc:407] Connection header received
0806 16:11:11.502247 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:11.502252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:11.502354 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:11.502480 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:11.504423 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.505130 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.506871 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.507265 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.507411 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.512991 (+ 5580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:11.512999 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:11.513005 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:11.513057 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:11.515599 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:11.516692 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:11.516697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:11.516698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:11.516825 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:11.517330 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:11.517339 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:11.517340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:11.517598 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:11.517735 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:11.518472 (+ 737us) server_negotiation.cc:299] Negotiation successful
0806 16:11:11.518702 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260806 16:11:11.520423 3071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.498957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:34937 (local address 127.8.49.2:36999)
0806 16:11:11.500037 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:11.500118 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:11.501900 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:11.502775 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:11.502786 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:11.503446 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:11.504185 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.504204 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.505313 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.505316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.506570 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.506585 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.507713 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.507724 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.508130 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.509347 (+ 1217us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:11.509373 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:11.512514 (+ 3141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:11.515864 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:11.515876 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:11.515904 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:11.516446 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:11.516973 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:11.516976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:11.516977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:11.517206 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:11.517734 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:11.517747 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:11.518139 (+ 392us) client_negotiation.cc:769] Sending connection context
0806 16:11:11.518414 (+ 275us) client_negotiation.cc:240] Negotiation successful
0806 16:11:11.518950 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":200,"threads_started":1}
WARNING: no policy specified for kudu/127.8.49.3@KRBTEST.COM; defaulting to no policy
I20260806 16:11:11.522668 3069 heartbeater.cc:347] Connected to a master server at 127.8.49.62:34937
I20260806 16:11:11.523073 3069 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:11.524148 3069 heartbeater.cc:511] Master 127.8.49.62:34937 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.3@KRBTEST.COM" created.
I20260806 16:11:11.526088 2653 ts_manager.cc:194] Registered new tserver with Master: 907af58cb6e64be68c6784a397371865 (127.8.49.2:36937)
I20260806 16:11:11.527289 2653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.2@KRBTEST.COM'} at 127.8.49.2:36999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
Entry for principal kudu/127.8.49.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab.
I20260806 16:11:11.553301 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34937
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:32931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:11.714526 3074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:11.714911 3074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:11.715009 3074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:11.719484 3074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:11.719580 3074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:11.719647 3074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:11:11.725847 3074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:32931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.3.keytab
--principal=kudu/127.8.49.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:34937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:11.727283 3074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:11.728555 3074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:11.742656 3081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:11.742697 3080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:11.742678 3085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:11.743371 3074 server_base.cc:1034] running on GCE node
I20260806 16:11:11.743960 3074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:11.744879 3074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:11.746393 3074 hybrid_clock.cc:648] HybridClock initialized: now 1786032671746341 us; error 75 us; skew 500 ppm
Aug 06 16:11:11 dist-test-slave-7z32 krb5kdc[2597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032671, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:11.750342 3074 init.cc:377] Logged in from keytab as kudu/127.8.49.3@KRBTEST.COM (short username kudu)
I20260806 16:11:11.751736 3074 webserver.cc:460] Webserver started at http://127.8.49.3:44711/ using document root <none> and password file <none>
I20260806 16:11:11.752496 3074 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:11.752615 3074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:11.752897 3074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:11.755609 3074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "d340b6bf0d3c4d0484145003110f84d5"
format_stamp: "Formatted at 2026-08-06 16:11:11 on dist-test-slave-7z32"
server_key: "4516c2336d5502b94e9771cf0932e6d6"
server_key_iv: "03b17a51aaf6b263fa2340cd62936db9"
server_key_version: "encryptionkey@0"
I20260806 16:11:11.756292 3074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "d340b6bf0d3c4d0484145003110f84d5"
format_stamp: "Formatted at 2026-08-06 16:11:11 on dist-test-slave-7z32"
server_key: "4516c2336d5502b94e9771cf0932e6d6"
server_key_iv: "03b17a51aaf6b263fa2340cd62936db9"
server_key_version: "encryptionkey@0"
I20260806 16:11:11.762274 3074 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 16:11:11.765949 3097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:11.767931 3074 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:11.768333 3074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "d340b6bf0d3c4d0484145003110f84d5"
format_stamp: "Formatted at 2026-08-06 16:11:11 on dist-test-slave-7z32"
server_key: "4516c2336d5502b94e9771cf0932e6d6"
server_key_iv: "03b17a51aaf6b263fa2340cd62936db9"
server_key_version: "encryptionkey@0"
I20260806 16:11:11.768610 3074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:11.787952 3074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:11.792928 3074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:11.793254 3074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:11.794035 3074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:11.795192 3074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:11.795295 3074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:11.795374 3074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:11.795395 3074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:11.808008 3074 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:38367
I20260806 16:11:11.808028 3240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:38367 every 8 connection(s)
I20260806 16:11:11.809150 3074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:11:11.816118 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 3074
I20260806 16:11:11.816337 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:11:11.816811 8388 external_mini_cluster.cc:1428] Setting key 6f3ce819477f289364bd5be52318ccfc
Aug 06 16:11:11 dist-test-slave-7z32 krb5kdc[2597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032671, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.3@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:11.832070 2699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.812365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:52509 (local address 127.8.49.62:34937)
0806 16:11:11.812882 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:11.812892 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:11.814027 (+ 1135us) server_negotiation.cc:407] Connection header received
0806 16:11:11.816333 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:11.816340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:11.816439 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:11.816576 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:11.819298 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.820118 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.821878 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.822162 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.822306 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.826244 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:11.826248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:11.826250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:11.826292 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:11.828326 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:11.829244 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:11.829248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:11.829249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:11.829372 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:11.830095 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:11.830098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:11.830099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:11.830336 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:11.830465 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:11.831667 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0806 16:11:11.831849 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260806 16:11:11.833839 3244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.812855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:34937 (local address 127.8.49.3:52509)
0806 16:11:11.813729 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:11.813814 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:11.815815 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:11.816918 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:11.816932 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:11.817589 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:11.818615 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.818644 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.820379 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.820383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.821653 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.821669 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.822424 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.822428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.822667 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.823766 (+ 1099us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:11.823791 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:11.825903 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:11.828553 (+ 2650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:11.828564 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:11.828591 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:11.829038 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:11.829628 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:11.829633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:11.829634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:11.829942 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:11.830610 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:11.830623 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:11.831193 (+ 570us) client_negotiation.cc:769] Sending connection context
0806 16:11:11.831547 (+ 354us) client_negotiation.cc:240] Negotiation successful
0806 16:11:11.832122 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":167,"threads_started":1}
I20260806 16:11:11.835783 3241 heartbeater.cc:347] Connected to a master server at 127.8.49.62:34937
I20260806 16:11:11.836149 3241 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:11.836961 3241 heartbeater.cc:511] Master 127.8.49.62:34937 requested a full tablet report, sending...
I20260806 16:11:11.838985 2653 ts_manager.cc:194] Registered new tserver with Master: d340b6bf0d3c4d0484145003110f84d5 (127.8.49.3:38367)
I20260806 16:11:11.839776 2653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.49.3@KRBTEST.COM'} at 127.8.49.3:52509
I20260806 16:11:11.846973 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:11:11.862703 2699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.850073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40348 (local address 127.8.49.62:34937)
0806 16:11:11.850275 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:11.850281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:11.850583 (+ 302us) server_negotiation.cc:407] Connection header received
0806 16:11:11.851145 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:11.851149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:11.851224 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:11.851443 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:11.852739 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.853324 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.855191 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.855562 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.855749 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.857967 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:11.857971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:11.857974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:11.858010 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:11.860115 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:11.860931 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:11.860934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:11.860936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:11.861041 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:11.861404 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:11.861406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:11.861407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:11.861768 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:11.861896 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:11.862310 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 16:11:11.862452 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 16:11:11.868074 2653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:11:11.871722 2653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:11:11.894508 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.885977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.8.49.1:35517)
0806 16:11:11.887513 (+ 1536us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:11.887526 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:11.887562 (+ 36us) server_negotiation.cc:407] Connection header received
0806 16:11:11.887671 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:11.887678 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:11.887995 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:11.888214 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:11.888725 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.890205 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.892459 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.893453 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.893911 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.894116 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:11.894182 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 16:11:11.894316 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1386,"thread_start_us":216,"threads_started":1}
I20260806 16:11:11.894578 3259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.885713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:35517 (local address 127.0.0.1:49106)
0806 16:11:11.886518 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:11.886619 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:11.886884 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:11.888199 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:11.888208 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:11.888257 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:11.888523 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.888531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.890376 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.890380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.892135 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.892149 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.893623 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.893628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.893793 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.893857 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 16:11:11.894003 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 16:11:11.894316 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":299,"spinlock_wait_cycles":6784,"thread_start_us":244,"threads_started":1}
I20260806 16:11:11.894999 3264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.887414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49356 (local address 127.8.49.3:38367)
0806 16:11:11.887855 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:11.887866 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:11.888740 (+ 874us) server_negotiation.cc:407] Connection header received
0806 16:11:11.888819 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:11.888824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:11.888980 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:11.889160 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:11.889392 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.890367 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.892581 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.893443 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.894430 (+ 987us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.894594 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:11.894651 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 16:11:11.894757 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":108,"threads_started":1}
I20260806 16:11:11.895751 3261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.886393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:36937 (local address 127.0.0.1:45788)
0806 16:11:11.886801 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:11.886852 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:11.886953 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:11.888203 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:11.888211 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:11.888264 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:11.888502 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.888510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.890377 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.890380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.892523 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.892536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.893581 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.893585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.895461 (+ 1876us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.895485 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:11:11.895558 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 16:11:11.895627 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":90,"spinlock_wait_cycles":5760,"thread_start_us":143,"threads_started":1}
I20260806 16:11:11.896123 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.886735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:38367 (local address 127.0.0.1:49356)
0806 16:11:11.888518 (+ 1783us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:11.888534 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:11.888634 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:11.889128 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:11.889131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:11.889150 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:11.889283 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.889290 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.890509 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.890513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.892408 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.892425 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.893796 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.893800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.893930 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.893962 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 16:11:11.894023 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 16:11:11.894277 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1712,"mutex_wait_us":133,"thread_start_us":140,"threads_started":1}
I20260806 16:11:11.896135 3263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.886976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45788 (local address 127.8.49.2:36937)
0806 16:11:11.887600 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:11.887614 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:11.887643 (+ 29us) server_negotiation.cc:407] Connection header received
0806 16:11:11.887748 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:11.887755 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:11.888008 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:11.888210 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:11.888708 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.890175 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.892666 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.893406 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.893569 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.893813 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:11.895669 (+ 1856us) server_negotiation.cc:299] Negotiation successful
0806 16:11:11.895838 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":119,"threads_started":1}
I20260806 16:11:11.899226 3159 tablet_service.cc:1467] Processing CreateTablet for tablet bf6073ebdeba45a4bf6c826bde085cd5 (DEFAULT_TABLE table=test-table [id=5342f759226f4863981cb5f5061ea3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:11.899221 2805 tablet_service.cc:1467] Processing CreateTablet for tablet bf6073ebdeba45a4bf6c826bde085cd5 (DEFAULT_TABLE table=test-table [id=5342f759226f4863981cb5f5061ea3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:11.900035 2984 tablet_service.cc:1467] Processing CreateTablet for tablet bf6073ebdeba45a4bf6c826bde085cd5 (DEFAULT_TABLE table=test-table [id=5342f759226f4863981cb5f5061ea3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:11.900622 3159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6073ebdeba45a4bf6c826bde085cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:11.900622 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6073ebdeba45a4bf6c826bde085cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:11.901341 2984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6073ebdeba45a4bf6c826bde085cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:11.909806 3265 tablet_bootstrap.cc:492] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489: Bootstrap starting.
I20260806 16:11:11.910712 3266 tablet_bootstrap.cc:492] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5: Bootstrap starting.
I20260806 16:11:11.913380 3267 tablet_bootstrap.cc:492] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: Bootstrap starting.
I20260806 16:11:11.913618 3266 tablet_bootstrap.cc:654] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:11.914983 3265 tablet_bootstrap.cc:654] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:11.915048 3266 log.cc:824] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:11.916236 3265 log.cc:824] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:11.917722 3266 tablet_bootstrap.cc:492] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5: No bootstrap required, opened a new log
I20260806 16:11:11.917953 3267 tablet_bootstrap.cc:654] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:11.918026 3266 ts_tablet_manager.cc:1397] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260806 16:11:11.919278 3267 log.cc:824] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:11.920198 3265 tablet_bootstrap.cc:492] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489: No bootstrap required, opened a new log
I20260806 16:11:11.920554 3265 ts_tablet_manager.cc:1397] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 16:11:11.922008 3266 raft_consensus.cc:348] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857c9e28469d47488446bfe618c0a489" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35517 } } peers { permanent_uuid: "d340b6bf0d3c4d0484145003110f84d5" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 38367 } } peers { permanent_uuid: "907af58cb6e64be68c6784a397371865" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36937 } }
I20260806 16:11:11.922358 3266 raft_consensus.cc:374] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:11.922366 3267 tablet_bootstrap.cc:492] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: No bootstrap required, opened a new log
I20260806 16:11:11.922429 3266 raft_consensus.cc:729] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d340b6bf0d3c4d0484145003110f84d5, State: Initialized, Role: FOLLOWER
I20260806 16:11:11.922657 3267 ts_tablet_manager.cc:1397] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260806 16:11:11.923005 3266 consensus_queue.cc:260] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857c9e28469d47488446bfe618c0a489" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35517 } } peers { permanent_uuid: "d340b6bf0d3c4d0484145003110f84d5" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 38367 } } peers { permanent_uuid: "907af58cb6e64be68c6784a397371865" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36937 } }
I20260806 16:11:11.923960 3241 heartbeater.cc:503] Master 127.8.49.62:34937 was elected leader, sending a full tablet report...
I20260806 16:11:11.924260 3266 ts_tablet_manager.cc:1428] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260806 16:11:11.928721 3267 raft_consensus.cc:348] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857c9e28469d47488446bfe618c0a489" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35517 } } peers { permanent_uuid: "d340b6bf0d3c4d0484145003110f84d5" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 38367 } } peers { permanent_uuid: "907af58cb6e64be68c6784a397371865" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36937 } }
I20260806 16:11:11.928913 3265 raft_consensus.cc:348] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857c9e28469d47488446bfe618c0a489" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35517 } } peers { permanent_uuid: "d340b6bf0d3c4d0484145003110f84d5" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 38367 } } peers { permanent_uuid: "907af58cb6e64be68c6784a397371865" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36937 } }
I20260806 16:11:11.929194 3265 raft_consensus.cc:374] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:11.929194 3267 raft_consensus.cc:374] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:11.929270 3265 raft_consensus.cc:729] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857c9e28469d47488446bfe618c0a489, State: Initialized, Role: FOLLOWER
I20260806 16:11:11.929270 3267 raft_consensus.cc:729] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907af58cb6e64be68c6784a397371865, State: Initialized, Role: FOLLOWER
I20260806 16:11:11.929940 3265 consensus_queue.cc:260] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857c9e28469d47488446bfe618c0a489" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35517 } } peers { permanent_uuid: "d340b6bf0d3c4d0484145003110f84d5" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 38367 } } peers { permanent_uuid: "907af58cb6e64be68c6784a397371865" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36937 } }
I20260806 16:11:11.929940 3267 consensus_queue.cc:260] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857c9e28469d47488446bfe618c0a489" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35517 } } peers { permanent_uuid: "d340b6bf0d3c4d0484145003110f84d5" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 38367 } } peers { permanent_uuid: "907af58cb6e64be68c6784a397371865" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36937 } }
I20260806 16:11:11.931257 3069 heartbeater.cc:503] Master 127.8.49.62:34937 was elected leader, sending a full tablet report...
I20260806 16:11:11.931561 3265 ts_tablet_manager.cc:1428] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489: Time spent starting tablet: real 0.011s user 0.006s sys 0.003s
I20260806 16:11:11.931577 3267 ts_tablet_manager.cc:1428] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: Time spent starting tablet: real 0.009s user 0.010s sys 0.001s
I20260806 16:11:11.932107 2893 heartbeater.cc:503] Master 127.8.49.62:34937 was elected leader, sending a full tablet report...
I20260806 16:11:11.952581 3276 raft_consensus.cc:482] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:11:11.952955 3276 raft_consensus.cc:504] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857c9e28469d47488446bfe618c0a489" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35517 } } peers { permanent_uuid: "d340b6bf0d3c4d0484145003110f84d5" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 38367 } } peers { permanent_uuid: "907af58cb6e64be68c6784a397371865" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36937 } }
I20260806 16:11:11.954713 3276 leader_election.cc:288] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d340b6bf0d3c4d0484145003110f84d5 (127.8.49.3:38367), 907af58cb6e64be68c6784a397371865 (127.8.49.2:36937)
I20260806 16:11:11.960534 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.955178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:36937 (local address 127.8.49.1:48539)
0806 16:11:11.955615 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:11.955636 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:11.955815 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:11.956302 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:11.956306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:11.956334 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:11.956559 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.956568 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.957684 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.957688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.958943 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.958955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.959995 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.959998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.960094 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.960123 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 16:11:11.960202 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 16:11:11.960317 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":180,"threads_started":1}
I20260806 16:11:11.960557 3263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.955340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:48539 (local address 127.8.49.2:36937)
0806 16:11:11.955522 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:11.955528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:11.955771 (+ 243us) server_negotiation.cc:407] Connection header received
0806 16:11:11.956005 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:11.956011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:11.956104 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:11.956213 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:11.956720 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.957544 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.959120 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.959809 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.959947 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.960049 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:11.960266 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 16:11:11.960382 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 16:11:11.960919 3264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.955310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:40831 (local address 127.8.49.3:38367)
0806 16:11:11.955689 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:11.955697 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:11.956230 (+ 533us) server_negotiation.cc:407] Connection header received
0806 16:11:11.956316 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:11.956319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:11.956386 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:11.956499 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:11.956928 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.957693 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.959468 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.960169 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.960334 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.960407 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:11.960659 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 16:11:11.960762 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269}
I20260806 16:11:11.960917 3278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.955185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:38367 (local address 127.8.49.1:40831)
0806 16:11:11.956001 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:11.956020 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:11.956175 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:11.956535 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:11.956538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:11.956558 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:11.956740 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.956748 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.957936 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.957939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.959311 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:11.959325 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.960367 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:11.960371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:11.960489 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:11.960514 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 16:11:11.960620 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 16:11:11.960721 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":6,"thread_start_us":173,"threads_started":1}
I20260806 16:11:11.961908 3181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6073ebdeba45a4bf6c826bde085cd5" candidate_uuid: "857c9e28469d47488446bfe618c0a489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d340b6bf0d3c4d0484145003110f84d5" is_pre_election: true
I20260806 16:11:11.961908 3015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6073ebdeba45a4bf6c826bde085cd5" candidate_uuid: "857c9e28469d47488446bfe618c0a489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907af58cb6e64be68c6784a397371865" is_pre_election: true
I20260806 16:11:11.962343 3181 raft_consensus.cc:2454] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 857c9e28469d47488446bfe618c0a489 in term 0.
I20260806 16:11:11.962343 3015 raft_consensus.cc:2454] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 857c9e28469d47488446bfe618c0a489 in term 0.
I20260806 16:11:11.963140 2749 leader_election.cc:302] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 857c9e28469d47488446bfe618c0a489, 907af58cb6e64be68c6784a397371865, d340b6bf0d3c4d0484145003110f84d5; no voters:
I20260806 16:11:11.963580 3276 raft_consensus.cc:2781] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:11:11.963728 3276 raft_consensus.cc:482] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:11:11.963771 3276 raft_consensus.cc:3037] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:11.965418 3276 raft_consensus.cc:504] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857c9e28469d47488446bfe618c0a489" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35517 } } peers { permanent_uuid: "d340b6bf0d3c4d0484145003110f84d5" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 38367 } } peers { permanent_uuid: "907af58cb6e64be68c6784a397371865" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36937 } }
I20260806 16:11:11.966078 3276 leader_election.cc:288] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [CANDIDATE]: Term 1 election: Requested vote from peers d340b6bf0d3c4d0484145003110f84d5 (127.8.49.3:38367), 907af58cb6e64be68c6784a397371865 (127.8.49.2:36937)
I20260806 16:11:11.966704 3181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6073ebdeba45a4bf6c826bde085cd5" candidate_uuid: "857c9e28469d47488446bfe618c0a489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d340b6bf0d3c4d0484145003110f84d5"
I20260806 16:11:11.966931 3181 raft_consensus.cc:3037] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:11.967015 3015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6073ebdeba45a4bf6c826bde085cd5" candidate_uuid: "857c9e28469d47488446bfe618c0a489" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907af58cb6e64be68c6784a397371865"
I20260806 16:11:11.967267 3015 raft_consensus.cc:3037] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:11.968449 3181 raft_consensus.cc:2454] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 857c9e28469d47488446bfe618c0a489 in term 1.
I20260806 16:11:11.969077 2745 leader_election.cc:302] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 857c9e28469d47488446bfe618c0a489, d340b6bf0d3c4d0484145003110f84d5; no voters:
I20260806 16:11:11.969142 3015 raft_consensus.cc:2454] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 857c9e28469d47488446bfe618c0a489 in term 1.
I20260806 16:11:11.969385 3276 raft_consensus.cc:2781] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:11.969730 3276 raft_consensus.cc:686] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [term 1 LEADER]: Becoming Leader. State: Replica: 857c9e28469d47488446bfe618c0a489, State: Running, Role: LEADER
I20260806 16:11:11.970288 3276 consensus_queue.cc:237] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857c9e28469d47488446bfe618c0a489" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35517 } } peers { permanent_uuid: "d340b6bf0d3c4d0484145003110f84d5" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 38367 } } peers { permanent_uuid: "907af58cb6e64be68c6784a397371865" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36937 } }
I20260806 16:11:11.975723 2652 catalog_manager.cc:5495] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 reported cstate change: term changed from 0 to 1, leader changed from <none> to 857c9e28469d47488446bfe618c0a489 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "857c9e28469d47488446bfe618c0a489" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857c9e28469d47488446bfe618c0a489" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d340b6bf0d3c4d0484145003110f84d5" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "907af58cb6e64be68c6784a397371865" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36937 } health_report { overall_health: UNKNOWN } } }
W20260806 16:11:11.997501 3070 tablet.cc:2369] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:11:12.003530 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:11.997413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49110 (local address 127.8.49.1:35517)
0806 16:11:11.997653 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:11.997662 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:11.997687 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:11:11.997853 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:11.997857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:11.997967 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:11.998105 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:11.998590 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:11.999402 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:12.000654 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:12.000927 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:12.001299 (+ 372us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:12.001972 (+ 673us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:12.002700 (+ 728us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:12.002887 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:12.003080 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 16:11:12.003246 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 16:11:12.012909 3015 raft_consensus.cc:1261] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865 [term 1 FOLLOWER]: Refusing update from remote peer 857c9e28469d47488446bfe618c0a489: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:12.013098 3181 raft_consensus.cc:1261] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5 [term 1 FOLLOWER]: Refusing update from remote peer 857c9e28469d47488446bfe618c0a489: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:12.014240 3279 consensus_queue.cc:1035] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d340b6bf0d3c4d0484145003110f84d5" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:12.014595 3276 consensus_queue.cc:1035] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489 [LEADER]: Connected to new peer: Peer: permanent_uuid: "907af58cb6e64be68c6784a397371865" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:12.024739 3282 mvcc.cc:205] Tried to move back new op lower bound from 7315589824553332736 to 7315589824394473472. Current Snapshot: MvccSnapshot[applied={T|T < 7315589824553332736}]
I20260806 16:11:12.026434 3283 mvcc.cc:205] Tried to move back new op lower bound from 7315589824553332736 to 7315589824394473472. Current Snapshot: MvccSnapshot[applied={T|T < 7315589824553332736}]
I20260806 16:11:12.027665 3285 mvcc.cc:205] Tried to move back new op lower bound from 7315589824553332736 to 7315589824394473472. Current Snapshot: MvccSnapshot[applied={T|T < 7315589824553332736}]
I20260806 16:11:12.038023 2645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:12.038287 2645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:12.042047 2645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d244413bc3094203922825f3dbb6c6c5: Sending DeleteTablet for 3 replicas of tablet bf6073ebdeba45a4bf6c826bde085cd5
I20260806 16:11:12.043480 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6073ebdeba45a4bf6c826bde085cd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:12 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:49106
I20260806 16:11:12.043550 2984 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6073ebdeba45a4bf6c826bde085cd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:12 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:45788
I20260806 16:11:12.043612 3159 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6073ebdeba45a4bf6c826bde085cd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:12 UTC) from {username='kudu', principal='kudu/127.8.49.62@KRBTEST.COM'} at 127.0.0.1:49356
I20260806 16:11:12.044566 3295 tablet_replica.cc:330] T bf6073ebdeba45a4bf6c826bde085cd5 P 857c9e28469d47488446bfe618c0a489: stopping tablet replica
I20260806 16:11:12.045037 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 2728
I20260806 16:11:12.056232 3297 tablet_replica.cc:330] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5: stopping tablet replica
I20260806 16:11:12.056421 3296 tablet_replica.cc:330] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: stopping tablet replica
I20260806 16:11:12.056913 3296 raft_consensus.cc:2229] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:11:12.056921 3297 raft_consensus.cc:2229] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:11:12.057488 3297 raft_consensus.cc:2258] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:12.057516 3296 raft_consensus.cc:2258] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:12.059900 3296 ts_tablet_manager.cc:1905] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:12.060222 3297 ts_tablet_manager.cc:1905] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:12.063145 3296 ts_tablet_manager.cc:1918] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:12.063310 3296 log.cc:1196] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/bf6073ebdeba45a4bf6c826bde085cd5
I20260806 16:11:12.063772 3296 ts_tablet_manager.cc:1939] T bf6073ebdeba45a4bf6c826bde085cd5 P 907af58cb6e64be68c6784a397371865: Deleting consensus metadata
I20260806 16:11:12.064330 3297 ts_tablet_manager.cc:1918] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:12.064498 3297 log.cc:1196] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/bf6073ebdeba45a4bf6c826bde085cd5
I20260806 16:11:12.064899 3297 ts_tablet_manager.cc:1939] T bf6073ebdeba45a4bf6c826bde085cd5 P d340b6bf0d3c4d0484145003110f84d5: Deleting consensus metadata
I20260806 16:11:12.066236 2637 catalog_manager.cc:4841] TS 907af58cb6e64be68c6784a397371865 (127.8.49.2:36937): tablet bf6073ebdeba45a4bf6c826bde085cd5 (table test-table [id=5342f759226f4863981cb5f5061ea3f5]) successfully deleted
I20260806 16:11:12.066632 2634 catalog_manager.cc:4841] TS d340b6bf0d3c4d0484145003110f84d5 (127.8.49.3:38367): tablet bf6073ebdeba45a4bf6c826bde085cd5 (table test-table [id=5342f759226f4863981cb5f5061ea3f5]) successfully deleted
W20260806 16:11:12.067560 2634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:35517 (error 108)
I20260806 16:11:12.069029 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 2900
I20260806 16:11:12.069262 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:12.068767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:35517 (local address 127.0.0.1:49126)
0806 16:11:12.069027 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:12.069135 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:35517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260806 16:11:12.091656 2634 catalog_manager.cc:4584] TS 857c9e28469d47488446bfe618c0a489 (127.8.49.1:35517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf6073ebdeba45a4bf6c826bde085cd5: Network error: Client connection negotiation failed: client connection to 127.8.49.1:35517: connect: Connection refused (error 111)
I20260806 16:11:12.098939 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 3074
I20260806 16:11:12.116979 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 2614
2026-08-06T16:11:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (3268 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T16:11:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:12Z Disabled control of system clock
I20260806 16:11:12.168807 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:39995
--webserver_interface=127.8.49.62
--webserver_port=0
--builtin_ntp_servers=127.8.49.20:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:39995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:11:12.325232 3302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:12.325708 3302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:12.325755 3302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:12.330524 3302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:12.330660 3302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:12.330681 3302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:12.330704 3302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:12.330724 3302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:12.336550 3302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:39995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:39995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:12.338402 3302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:12.340485 3302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:12.352835 3308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:12.352839 3309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:12.353183 3302 server_base.cc:1034] running on GCE node
W20260806 16:11:12.352897 3313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:12.354337 3302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:12.356040 3302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:12.357331 3302 hybrid_clock.cc:648] HybridClock initialized: now 1786032672357283 us; error 63 us; skew 500 ppm
I20260806 16:11:12.360354 3302 webserver.cc:460] Webserver started at http://127.8.49.62:46505/ using document root <none> and password file <none>
I20260806 16:11:12.361357 3302 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:12.361613 3302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:12.361922 3302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:12.365365 3302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "8fb0fed6c16c48babf669f07cc391ccf"
format_stamp: "Formatted at 2026-08-06 16:11:12 on dist-test-slave-7z32"
server_key: "3193b29012d0198a972ed6f402941903"
server_key_iv: "51e632946882a5caa74d539013c98c70"
server_key_version: "encryptionkey@0"
I20260806 16:11:12.366544 3302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "8fb0fed6c16c48babf669f07cc391ccf"
format_stamp: "Formatted at 2026-08-06 16:11:12 on dist-test-slave-7z32"
server_key: "3193b29012d0198a972ed6f402941903"
server_key_iv: "51e632946882a5caa74d539013c98c70"
server_key_version: "encryptionkey@0"
I20260806 16:11:12.372575 3302 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260806 16:11:12.376540 3319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:12.378767 3302 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:11:12.379065 3302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "8fb0fed6c16c48babf669f07cc391ccf"
format_stamp: "Formatted at 2026-08-06 16:11:12 on dist-test-slave-7z32"
server_key: "3193b29012d0198a972ed6f402941903"
server_key_iv: "51e632946882a5caa74d539013c98c70"
server_key_version: "encryptionkey@0"
I20260806 16:11:12.379258 3302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:12.406625 3302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:12.408023 3302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:12.408334 3302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:12.421104 3302 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:39995
I20260806 16:11:12.421168 3393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:39995 every 8 connection(s)
I20260806 16:11:12.422896 3302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:12.428716 3394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:12.431375 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 3302
I20260806 16:11:12.431528 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:12.431878 8388 external_mini_cluster.cc:1428] Setting key 1bb998ba38fa33a0bd04fcde28be3329
I20260806 16:11:12.444015 3394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf: Bootstrap starting.
I20260806 16:11:12.448017 3394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:12.449030 3394 log.cc:824] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:12.450708 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:12.434045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34128 (local address 127.8.49.62:39995)
0806 16:11:12.435233 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:12.435259 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:12.435319 (+ 60us) server_negotiation.cc:407] Connection header received
0806 16:11:12.436655 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:12.436699 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:12.437473 (+ 774us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:12.438008 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:12.439393 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:12.441508 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:12.442597 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:12.443778 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:12.444259 (+ 481us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:12.446153 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:12.446167 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:12.446202 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:12.447140 (+ 938us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:12.447267 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:12.447322 (+ 55us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:12.447512 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:12.448896 (+ 1384us) server_negotiation.cc:299] Negotiation successful
0806 16:11:12.449346 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":256,"threads_started":1}
I20260806 16:11:12.452015 3394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf: No bootstrap required, opened a new log
I20260806 16:11:12.457283 3394 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb0fed6c16c48babf669f07cc391ccf" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39995 } }
I20260806 16:11:12.457749 3394 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:12.457850 3394 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb0fed6c16c48babf669f07cc391ccf, State: Initialized, Role: FOLLOWER
I20260806 16:11:12.458446 3394 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb0fed6c16c48babf669f07cc391ccf" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39995 } }
I20260806 16:11:12.458626 3394 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:12.458721 3394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:12.458878 3394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:12.460224 3394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb0fed6c16c48babf669f07cc391ccf" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39995 } }
I20260806 16:11:12.460708 3394 leader_election.cc:302] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fb0fed6c16c48babf669f07cc391ccf; no voters:
I20260806 16:11:12.461181 3394 leader_election.cc:288] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:12.461740 3402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:12.462059 3402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [term 1 LEADER]: Becoming Leader. State: Replica: 8fb0fed6c16c48babf669f07cc391ccf, State: Running, Role: LEADER
I20260806 16:11:12.462555 3402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb0fed6c16c48babf669f07cc391ccf" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39995 } }
I20260806 16:11:12.462725 3394 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:12.465132 3404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fb0fed6c16c48babf669f07cc391ccf. Latest consensus state: current_term: 1 leader_uuid: "8fb0fed6c16c48babf669f07cc391ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb0fed6c16c48babf669f07cc391ccf" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39995 } } }
I20260806 16:11:12.465216 3403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fb0fed6c16c48babf669f07cc391ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb0fed6c16c48babf669f07cc391ccf" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39995 } } }
I20260806 16:11:12.465341 3404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:12.465389 3403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:12.465808 3427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:12.470701 3427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:12.478492 3427 catalog_manager.cc:1347] Generated new cluster ID: cc2dc39c37e54517838a23f4a12e568a
I20260806 16:11:12.478627 3427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:12.513868 3427 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:12.515177 3427 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:12.526841 3427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf: Generated new TSK 0
I20260806 16:11:12.528128 3427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:12.543505 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.1:0
--local_ip_for_outbound_sockets=127.8.49.1
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39995
--builtin_ntp_servers=127.8.49.20:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:11:12.686722 3432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:12.687148 3432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:12.687233 3432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:12.692363 3432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:12.692516 3432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:12.692626 3432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.1
I20260806 16:11:12.698136 3432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.49.1
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.1
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:12.699792 3432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:12.701414 3432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:12.713353 3441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:12.713446 3437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:12.713344 3439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:12.713788 3432 server_base.cc:1034] running on GCE node
I20260806 16:11:12.714468 3432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:12.715626 3432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:12.716885 3432 hybrid_clock.cc:648] HybridClock initialized: now 1786032672716837 us; error 68 us; skew 500 ppm
I20260806 16:11:12.720120 3432 webserver.cc:460] Webserver started at http://127.8.49.1:36195/ using document root <none> and password file <none>
I20260806 16:11:12.721043 3432 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:12.721174 3432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:12.721626 3432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:12.725013 3432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/data/instance:
uuid: "29cecd30709e4b02b07cce67892c2678"
format_stamp: "Formatted at 2026-08-06 16:11:12 on dist-test-slave-7z32"
server_key: "d316be8335567f260435d1947bb2b334"
server_key_iv: "08d57c52878c1b52e7229f92bf633ce5"
server_key_version: "encryptionkey@0"
I20260806 16:11:12.725952 3432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance:
uuid: "29cecd30709e4b02b07cce67892c2678"
format_stamp: "Formatted at 2026-08-06 16:11:12 on dist-test-slave-7z32"
server_key: "d316be8335567f260435d1947bb2b334"
server_key_iv: "08d57c52878c1b52e7229f92bf633ce5"
server_key_version: "encryptionkey@0"
I20260806 16:11:12.732069 3432 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:11:12.736410 3450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:12.738624 3432 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260806 16:11:12.738994 3432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/wal
uuid: "29cecd30709e4b02b07cce67892c2678"
format_stamp: "Formatted at 2026-08-06 16:11:12 on dist-test-slave-7z32"
server_key: "d316be8335567f260435d1947bb2b334"
server_key_iv: "08d57c52878c1b52e7229f92bf633ce5"
server_key_version: "encryptionkey@0"
I20260806 16:11:12.739199 3432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:12.762894 3432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:12.764266 3432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:12.764804 3432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:12.765758 3432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:12.767242 3432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:12.767329 3432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:12.767410 3432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:12.767463 3432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:12.781986 3432 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.1:41257
I20260806 16:11:12.782008 3606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.1:41257 every 8 connection(s)
I20260806 16:11:12.783565 3432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/data/info.pb
I20260806 16:11:12.787061 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 3432
I20260806 16:11:12.787251 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-0/wal/instance
I20260806 16:11:12.787599 8388 external_mini_cluster.cc:1428] Setting key f93c94a91f7c550c2e1ffbbe5198991e
I20260806 16:11:12.790009 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.2:0
--local_ip_for_outbound_sockets=127.8.49.2
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39995
--builtin_ntp_servers=127.8.49.20:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:11:12.797734 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:12.786492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:53463 (local address 127.8.49.62:39995)
0806 16:11:12.786759 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:12.786766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:12.788864 (+ 2098us) server_negotiation.cc:407] Connection header received
0806 16:11:12.790796 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:12.790800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:12.790953 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:12.791078 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:12.792625 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:12.793266 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:12.794734 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:12.795074 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:12.795212 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:12.796999 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:12.797004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:12.797007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:12.797028 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:12.797058 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:12.797068 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:12.797166 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:12.797400 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 16:11:12.797509 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260806 16:11:12.799357 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:12.787111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:39995 (local address 127.8.49.1:53463)
0806 16:11:12.788624 (+ 1513us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:12.788706 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:12.790484 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:12.791353 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:12.791363 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:12.791485 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:12.792416 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:12.792435 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:12.793459 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:12.793462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:12.794541 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:12.794553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:12.795300 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:12.795305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:12.795520 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:12.796461 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:11:12.796463 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:12.796526 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:11:12.796634 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:12.797183 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:12.797197 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:12.797207 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 16:11:12.797390 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 16:11:12.797861 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":501,"threads_started":1}
I20260806 16:11:12.801537 3607 heartbeater.cc:347] Connected to a master server at 127.8.49.62:39995
I20260806 16:11:12.801951 3607 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:12.802934 3607 heartbeater.cc:511] Master 127.8.49.62:39995 requested a full tablet report, sending...
I20260806 16:11:12.805436 3332 ts_manager.cc:194] Registered new tserver with Master: 29cecd30709e4b02b07cce67892c2678 (127.8.49.1:41257)
I20260806 16:11:12.807366 3332 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.1:53463
W20260806 16:11:12.918005 3611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:12.918347 3611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:12.918404 3611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:12.923177 3611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:12.923286 3611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:12.923348 3611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.2
I20260806 16:11:12.928867 3611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.49.2
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.2
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:12.930457 3611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:12.932020 3611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:12.944854 3617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:12.944767 3618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:12.944793 3621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:12.945190 3611 server_base.cc:1034] running on GCE node
I20260806 16:11:12.945953 3611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:12.947048 3611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:12.948284 3611 hybrid_clock.cc:648] HybridClock initialized: now 1786032672948257 us; error 58 us; skew 500 ppm
I20260806 16:11:12.950814 3611 webserver.cc:460] Webserver started at http://127.8.49.2:43785/ using document root <none> and password file <none>
I20260806 16:11:12.951615 3611 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:12.951694 3611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:12.951942 3611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:12.954316 3611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/data/instance:
uuid: "7f31b4e268c04b8b9967d6130beb02fe"
format_stamp: "Formatted at 2026-08-06 16:11:12 on dist-test-slave-7z32"
server_key: "f383b806b6518ea9c7343e424cfe270f"
server_key_iv: "2f75e2fcd4f2c0faf7661c366f584160"
server_key_version: "encryptionkey@0"
I20260806 16:11:12.955034 3611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance:
uuid: "7f31b4e268c04b8b9967d6130beb02fe"
format_stamp: "Formatted at 2026-08-06 16:11:12 on dist-test-slave-7z32"
server_key: "f383b806b6518ea9c7343e424cfe270f"
server_key_iv: "2f75e2fcd4f2c0faf7661c366f584160"
server_key_version: "encryptionkey@0"
I20260806 16:11:12.960103 3611 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:11:12.963991 3630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:12.966090 3611 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:11:12.966253 3611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/wal
uuid: "7f31b4e268c04b8b9967d6130beb02fe"
format_stamp: "Formatted at 2026-08-06 16:11:12 on dist-test-slave-7z32"
server_key: "f383b806b6518ea9c7343e424cfe270f"
server_key_iv: "2f75e2fcd4f2c0faf7661c366f584160"
server_key_version: "encryptionkey@0"
I20260806 16:11:12.966498 3611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:12.984226 3611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:12.985091 3611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:12.985329 3611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:12.986037 3611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:12.987197 3611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:12.987257 3611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:12.987305 3611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:12.987323 3611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:12.999053 3611 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.2:34741
I20260806 16:11:12.999205 3783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.2:34741 every 8 connection(s)
I20260806 16:11:13.000427 3611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/data/info.pb
I20260806 16:11:13.010670 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 3611
I20260806 16:11:13.010800 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/wal/instance
I20260806 16:11:13.011168 8388 external_mini_cluster.cc:1428] Setting key d9a9922c9c7ba483ed1e146866d40d25
I20260806 16:11:13.014137 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.49.3:0
--local_ip_for_outbound_sockets=127.8.49.3
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39995
--builtin_ntp_servers=127.8.49.20:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:11:13.014286 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.003405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.2:42925 (local address 127.8.49.62:39995)
0806 16:11:13.003695 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:13.003699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:13.005359 (+ 1660us) server_negotiation.cc:407] Connection header received
0806 16:11:13.007159 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:13.007163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:13.007234 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:13.007351 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:13.009200 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.009791 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.011438 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.011686 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.011844 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.013637 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:13.013641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:13.013644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:13.013659 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:13.013689 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:13.013698 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:13.013772 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:13.014058 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 16:11:13.014128 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 16:11:13.016000 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.003764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:39995 (local address 127.8.49.2:42925)
0806 16:11:13.005105 (+ 1341us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:13.005169 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:13.006857 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:13.007765 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:13.007778 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:13.007941 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:13.009005 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.009027 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.009959 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.009963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.011258 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.011274 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.011953 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.011958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.012311 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.013206 (+ 895us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:11:13.013208 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:13.013291 (+ 83us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:11:13.013416 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:13.013817 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:13.013830 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:13.013841 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 16:11:13.014025 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 16:11:13.014487 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":356,"threads_started":1}
I20260806 16:11:13.017884 3784 heartbeater.cc:347] Connected to a master server at 127.8.49.62:39995
I20260806 16:11:13.018281 3784 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:13.019155 3784 heartbeater.cc:511] Master 127.8.49.62:39995 requested a full tablet report, sending...
I20260806 16:11:13.021009 3332 ts_manager.cc:194] Registered new tserver with Master: 7f31b4e268c04b8b9967d6130beb02fe (127.8.49.2:34741)
I20260806 16:11:13.021998 3332 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.2:42925
W20260806 16:11:13.144562 3788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:13.144905 3788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:13.144948 3788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:13.149441 3788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:13.149540 3788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:13.149608 3788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.49.3
I20260806 16:11:13.155154 3788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.49.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.49.3
--webserver_port=0
--tserver_master_addrs=127.8.49.62:39995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.49.3
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:13.156481 3788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:13.158051 3788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:11:13.171387 3788 server_base.cc:1034] running on GCE node
W20260806 16:11:13.171345 3795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:13.171305 3800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:13.171326 3794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:13.173372 3788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:13.174319 3788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:13.177670 3788 hybrid_clock.cc:648] HybridClock initialized: now 1786032673177606 us; error 72 us; skew 500 ppm
I20260806 16:11:13.181079 3788 webserver.cc:460] Webserver started at http://127.8.49.3:44123/ using document root <none> and password file <none>
I20260806 16:11:13.182200 3788 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:13.182312 3788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:13.182585 3788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:13.186031 3788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/data/instance:
uuid: "41a27cf20f824030a9e525b502ed4369"
format_stamp: "Formatted at 2026-08-06 16:11:13 on dist-test-slave-7z32"
server_key: "eaa3ae1680cfd566104f7e16238f77ed"
server_key_iv: "7bff3ca77b275ca9b09a9cfce66d8dc7"
server_key_version: "encryptionkey@0"
I20260806 16:11:13.186784 3788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance:
uuid: "41a27cf20f824030a9e525b502ed4369"
format_stamp: "Formatted at 2026-08-06 16:11:13 on dist-test-slave-7z32"
server_key: "eaa3ae1680cfd566104f7e16238f77ed"
server_key_iv: "7bff3ca77b275ca9b09a9cfce66d8dc7"
server_key_version: "encryptionkey@0"
I20260806 16:11:13.192413 3788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:11:13.196568 3813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:13.198714 3788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 16:11:13.199015 3788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/wal
uuid: "41a27cf20f824030a9e525b502ed4369"
format_stamp: "Formatted at 2026-08-06 16:11:13 on dist-test-slave-7z32"
server_key: "eaa3ae1680cfd566104f7e16238f77ed"
server_key_iv: "7bff3ca77b275ca9b09a9cfce66d8dc7"
server_key_version: "encryptionkey@0"
I20260806 16:11:13.199203 3788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:13.219813 3788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:13.220726 3788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:13.220979 3788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:13.221709 3788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:11:13.222987 3788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:11:13.223078 3788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:13.223155 3788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:11:13.223210 3788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:13.236909 3788 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.3:46717
I20260806 16:11:13.236963 3983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.3:46717 every 8 connection(s)
I20260806 16:11:13.238118 3788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/data/info.pb
I20260806 16:11:13.246485 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 3788
I20260806 16:11:13.246706 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/wal/instance
I20260806 16:11:13.247179 8388 external_mini_cluster.cc:1428] Setting key c089843caae5ff4c3a65543c09a55dc7
I20260806 16:11:13.252861 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.241293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.3:57141 (local address 127.8.49.62:39995)
0806 16:11:13.241575 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:13.241580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:13.243194 (+ 1614us) server_negotiation.cc:407] Connection header received
0806 16:11:13.245194 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:13.245198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:13.245271 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:13.245523 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:13.247534 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.248247 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.249903 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.250143 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.250259 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.252159 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:13.252163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:13.252165 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:13.252181 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:13.252202 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:13.252211 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:13.252300 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:13.252569 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 16:11:13.252645 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260806 16:11:13.253911 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.241757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:39995 (local address 127.8.49.3:57141)
0806 16:11:13.242904 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:13.242987 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:13.244783 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:13.245886 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:13.245897 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:13.246122 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:13.247272 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.247305 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.248468 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.248473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.249747 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.249764 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.250336 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.250342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.250664 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.251768 (+ 1104us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:11:13.251773 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:13.251825 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:11:13.251915 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:13.252317 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:13.252328 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:13.252337 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:11:13.252558 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 16:11:13.252930 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":214,"threads_started":1}
I20260806 16:11:13.256193 3984 heartbeater.cc:347] Connected to a master server at 127.8.49.62:39995
I20260806 16:11:13.256721 3984 heartbeater.cc:464] Registering TS with master...
I20260806 16:11:13.257596 3984 heartbeater.cc:511] Master 127.8.49.62:39995 requested a full tablet report, sending...
I20260806 16:11:13.259323 3332 ts_manager.cc:194] Registered new tserver with Master: 41a27cf20f824030a9e525b502ed4369 (127.8.49.3:46717)
I20260806 16:11:13.260251 3332 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.49.3:57141
I20260806 16:11:13.263693 8388 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:11:13.272998 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.266611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34130 (local address 127.8.49.62:39995)
0806 16:11:13.266795 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:13.266800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:13.267167 (+ 367us) server_negotiation.cc:407] Connection header received
0806 16:11:13.267290 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:13.267293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:13.267349 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:13.267456 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:13.267833 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.268385 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.269561 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.269816 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.269971 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.272133 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:13.272139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:13.272220 (+ 81us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:13.272253 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:13.272285 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:13.272308 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:13.272409 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:13.272756 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 16:11:13.272843 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:11:13.278198 3332 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:11:13.281931 3332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:11:13.302825 3999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.294989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:34741 (local address 127.0.0.1:59080)
0806 16:11:13.295468 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:13.295527 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:13.295681 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:13.296676 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:13.296687 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:13.296737 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:13.296957 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.296967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.298722 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.298726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.300287 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.300307 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.302040 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.302043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.302152 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.302193 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 16:11:13.302372 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 16:11:13.302575 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":39,"thread_start_us":152,"threads_started":1}
I20260806 16:11:13.302979 4000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.295207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59080 (local address 127.8.49.2:34741)
0806 16:11:13.295734 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:13.295748 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:13.295785 (+ 37us) server_negotiation.cc:407] Connection header received
0806 16:11:13.295885 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:13.295916 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:13.296234 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:13.296497 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:13.297133 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.298545 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.300455 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.301516 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.301661 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.301971 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:13.302434 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 16:11:13.302583 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":243,"threads_started":1}
I20260806 16:11:13.303718 4001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.295952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:46717 (local address 127.0.0.1:55022)
0806 16:11:13.296373 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:13.296394 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:13.296510 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:13.297750 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:13.297754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:13.297776 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:13.297895 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.297902 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.299530 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.299534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.301370 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.301385 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.303139 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.303144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.303259 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.303301 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 16:11:13.303386 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 16:11:13.303494 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":43,"thread_start_us":152,"threads_started":1}
I20260806 16:11:13.303911 4005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.297153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41257 (local address 127.0.0.1:37846)
0806 16:11:13.297453 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:13.297469 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:13.297607 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:13.298568 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:13.298571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:13.298594 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:13.298755 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.298765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.300207 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.300210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.301599 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.301608 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.303410 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.303415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.303500 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.303521 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 16:11:13.303597 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 16:11:13.303661 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":3,"thread_start_us":106,"threads_started":1}
I20260806 16:11:13.304551 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.296775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37846 (local address 127.8.49.1:41257)
0806 16:11:13.297891 (+ 1116us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:13.297899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:13.297964 (+ 65us) server_negotiation.cc:407] Connection header received
0806 16:11:13.298079 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:13.298087 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:13.298390 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:13.298603 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:13.298929 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.299911 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.301733 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.302685 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.302814 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.303269 (+ 455us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:13.304230 (+ 961us) server_negotiation.cc:299] Negotiation successful
0806 16:11:13.304385 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":979,"thread_start_us":331,"threads_started":1}
I20260806 16:11:13.305032 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.296269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.8.49.3:46717)
0806 16:11:13.296943 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:13.296958 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:13.296987 (+ 29us) server_negotiation.cc:407] Connection header received
0806 16:11:13.297092 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:13.297102 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:13.297371 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:13.297561 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:13.298049 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.299352 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.301588 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.302686 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.302814 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.304361 (+ 1547us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:13.304441 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 16:11:13.304598 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":358,"threads_started":1}
I20260806 16:11:13.308514 3688 tablet_service.cc:1467] Processing CreateTablet for tablet cc23c49646424fe3933a72b1c4c086ea (DEFAULT_TABLE table=test-table [id=aa98e648b3ac43d38d710367a8562415]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:13.309370 3887 tablet_service.cc:1467] Processing CreateTablet for tablet cc23c49646424fe3933a72b1c4c086ea (DEFAULT_TABLE table=test-table [id=aa98e648b3ac43d38d710367a8562415]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:13.309708 3513 tablet_service.cc:1467] Processing CreateTablet for tablet cc23c49646424fe3933a72b1c4c086ea (DEFAULT_TABLE table=test-table [id=aa98e648b3ac43d38d710367a8562415]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:11:13.310365 3688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc23c49646424fe3933a72b1c4c086ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:13.310400 3887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc23c49646424fe3933a72b1c4c086ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:13.311561 3513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc23c49646424fe3933a72b1c4c086ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:13.321120 4006 tablet_bootstrap.cc:492] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: Bootstrap starting.
I20260806 16:11:13.321120 4008 tablet_bootstrap.cc:492] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: Bootstrap starting.
I20260806 16:11:13.322978 4009 tablet_bootstrap.cc:492] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678: Bootstrap starting.
I20260806 16:11:13.324182 4008 tablet_bootstrap.cc:654] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:13.324182 4006 tablet_bootstrap.cc:654] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:13.325248 4006 log.cc:824] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:13.325256 4008 log.cc:824] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:13.326797 4009 tablet_bootstrap.cc:654] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:13.327818 4006 tablet_bootstrap.cc:492] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: No bootstrap required, opened a new log
I20260806 16:11:13.328087 4006 ts_tablet_manager.cc:1397] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 16:11:13.328047 4009 log.cc:824] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:13.328804 4008 tablet_bootstrap.cc:492] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: No bootstrap required, opened a new log
I20260806 16:11:13.329099 4008 ts_tablet_manager.cc:1397] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 16:11:13.331054 4009 tablet_bootstrap.cc:492] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678: No bootstrap required, opened a new log
I20260806 16:11:13.331329 4009 ts_tablet_manager.cc:1397] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260806 16:11:13.332084 4006 raft_consensus.cc:348] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f31b4e268c04b8b9967d6130beb02fe" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34741 } } peers { permanent_uuid: "41a27cf20f824030a9e525b502ed4369" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46717 } } peers { permanent_uuid: "29cecd30709e4b02b07cce67892c2678" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41257 } }
I20260806 16:11:13.332386 4006 raft_consensus.cc:374] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:13.332451 4006 raft_consensus.cc:729] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f31b4e268c04b8b9967d6130beb02fe, State: Initialized, Role: FOLLOWER
I20260806 16:11:13.332945 4006 consensus_queue.cc:260] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f31b4e268c04b8b9967d6130beb02fe" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34741 } } peers { permanent_uuid: "41a27cf20f824030a9e525b502ed4369" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46717 } } peers { permanent_uuid: "29cecd30709e4b02b07cce67892c2678" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41257 } }
I20260806 16:11:13.334247 4006 ts_tablet_manager.cc:1428] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 16:11:13.334724 3784 heartbeater.cc:503] Master 127.8.49.62:39995 was elected leader, sending a full tablet report...
I20260806 16:11:13.335738 4008 raft_consensus.cc:348] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f31b4e268c04b8b9967d6130beb02fe" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34741 } } peers { permanent_uuid: "41a27cf20f824030a9e525b502ed4369" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46717 } } peers { permanent_uuid: "29cecd30709e4b02b07cce67892c2678" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41257 } }
I20260806 16:11:13.336272 4008 raft_consensus.cc:374] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:13.336337 4008 raft_consensus.cc:729] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a27cf20f824030a9e525b502ed4369, State: Initialized, Role: FOLLOWER
I20260806 16:11:13.337005 4008 consensus_queue.cc:260] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f31b4e268c04b8b9967d6130beb02fe" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34741 } } peers { permanent_uuid: "41a27cf20f824030a9e525b502ed4369" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46717 } } peers { permanent_uuid: "29cecd30709e4b02b07cce67892c2678" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41257 } }
I20260806 16:11:13.337675 4009 raft_consensus.cc:348] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f31b4e268c04b8b9967d6130beb02fe" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34741 } } peers { permanent_uuid: "41a27cf20f824030a9e525b502ed4369" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46717 } } peers { permanent_uuid: "29cecd30709e4b02b07cce67892c2678" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41257 } }
I20260806 16:11:13.338001 4009 raft_consensus.cc:374] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:13.338053 4009 raft_consensus.cc:729] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29cecd30709e4b02b07cce67892c2678, State: Initialized, Role: FOLLOWER
I20260806 16:11:13.338074 3984 heartbeater.cc:503] Master 127.8.49.62:39995 was elected leader, sending a full tablet report...
I20260806 16:11:13.338618 4008 ts_tablet_manager.cc:1428] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20260806 16:11:13.338699 4009 consensus_queue.cc:260] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f31b4e268c04b8b9967d6130beb02fe" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34741 } } peers { permanent_uuid: "41a27cf20f824030a9e525b502ed4369" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46717 } } peers { permanent_uuid: "29cecd30709e4b02b07cce67892c2678" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41257 } }
I20260806 16:11:13.339830 3607 heartbeater.cc:503] Master 127.8.49.62:39995 was elected leader, sending a full tablet report...
I20260806 16:11:13.340301 4009 ts_tablet_manager.cc:1428] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260806 16:11:13.460668 4019 raft_consensus.cc:482] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:11:13.460988 4019 raft_consensus.cc:504] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f31b4e268c04b8b9967d6130beb02fe" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34741 } } peers { permanent_uuid: "41a27cf20f824030a9e525b502ed4369" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46717 } } peers { permanent_uuid: "29cecd30709e4b02b07cce67892c2678" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41257 } }
I20260806 16:11:13.462775 4019 leader_election.cc:288] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f31b4e268c04b8b9967d6130beb02fe (127.8.49.2:34741), 41a27cf20f824030a9e525b502ed4369 (127.8.49.3:46717)
I20260806 16:11:13.470156 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.465290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:57517 (local address 127.8.49.3:46717)
0806 16:11:13.465455 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:13.465462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:13.465781 (+ 319us) server_negotiation.cc:407] Connection header received
0806 16:11:13.465917 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:13.465920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:13.465977 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:13.466084 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:13.466703 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.467333 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.468901 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.469360 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.469610 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.469693 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:13.469874 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 16:11:13.469967 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 16:11:13.470165 4000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.463522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.49.1:43353 (local address 127.8.49.2:34741)
0806 16:11:13.463719 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:13.463726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:13.465209 (+ 1483us) server_negotiation.cc:407] Connection header received
0806 16:11:13.465588 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:13.465596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:13.465702 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:13.465800 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:11:13.466297 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.467273 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.468857 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.469329 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.469458 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.469583 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:13.469845 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 16:11:13.469952 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 16:11:13.470219 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.465094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.3:46717 (local address 127.8.49.1:57517)
0806 16:11:13.465564 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:13.465583 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:13.465785 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:13.466227 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:13.466234 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:13.466259 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:13.466400 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.466407 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.467533 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.467536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.468605 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.468617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.469571 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.469575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.469697 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.469720 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:11:13.469833 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 16:11:13.469970 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":41,"thread_start_us":163,"threads_started":1}
I20260806 16:11:13.470714 4020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.463407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:34741 (local address 127.8.49.1:43353)
0806 16:11:13.465033 (+ 1626us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:13.465053 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:13.465335 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:13.465861 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:13.465867 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:13.465896 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:11:13.466104 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.466113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.467422 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.467426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.468590 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:13.468604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.469539 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.469543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:13.469667 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.469695 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 16:11:13.469791 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 16:11:13.469967 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1490,"mutex_wait_us":103,"thread_start_us":1204,"threads_started":1}
I20260806 16:11:13.471627 3915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc23c49646424fe3933a72b1c4c086ea" candidate_uuid: "29cecd30709e4b02b07cce67892c2678" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a27cf20f824030a9e525b502ed4369" is_pre_election: true
I20260806 16:11:13.471760 3714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc23c49646424fe3933a72b1c4c086ea" candidate_uuid: "29cecd30709e4b02b07cce67892c2678" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f31b4e268c04b8b9967d6130beb02fe" is_pre_election: true
I20260806 16:11:13.472070 3915 raft_consensus.cc:2454] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29cecd30709e4b02b07cce67892c2678 in term 0.
I20260806 16:11:13.472267 3714 raft_consensus.cc:2454] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29cecd30709e4b02b07cce67892c2678 in term 0.
I20260806 16:11:13.472728 3453 leader_election.cc:302] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29cecd30709e4b02b07cce67892c2678, 41a27cf20f824030a9e525b502ed4369; no voters:
I20260806 16:11:13.473206 4019 raft_consensus.cc:2781] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:11:13.473351 4019 raft_consensus.cc:482] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:11:13.473459 4019 raft_consensus.cc:3037] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:13.475083 4019 raft_consensus.cc:504] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f31b4e268c04b8b9967d6130beb02fe" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34741 } } peers { permanent_uuid: "41a27cf20f824030a9e525b502ed4369" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46717 } } peers { permanent_uuid: "29cecd30709e4b02b07cce67892c2678" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41257 } }
I20260806 16:11:13.475610 4019 leader_election.cc:288] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [CANDIDATE]: Term 1 election: Requested vote from peers 7f31b4e268c04b8b9967d6130beb02fe (127.8.49.2:34741), 41a27cf20f824030a9e525b502ed4369 (127.8.49.3:46717)
I20260806 16:11:13.476408 3714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc23c49646424fe3933a72b1c4c086ea" candidate_uuid: "29cecd30709e4b02b07cce67892c2678" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f31b4e268c04b8b9967d6130beb02fe"
I20260806 16:11:13.476416 3915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc23c49646424fe3933a72b1c4c086ea" candidate_uuid: "29cecd30709e4b02b07cce67892c2678" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a27cf20f824030a9e525b502ed4369"
I20260806 16:11:13.476619 3915 raft_consensus.cc:3037] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:13.476617 3714 raft_consensus.cc:3037] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:13.478232 3714 raft_consensus.cc:2454] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29cecd30709e4b02b07cce67892c2678 in term 1.
I20260806 16:11:13.478232 3915 raft_consensus.cc:2454] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29cecd30709e4b02b07cce67892c2678 in term 1.
I20260806 16:11:13.478713 3453 leader_election.cc:302] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29cecd30709e4b02b07cce67892c2678, 7f31b4e268c04b8b9967d6130beb02fe; no voters:
I20260806 16:11:13.479120 4019 raft_consensus.cc:2781] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:13.479540 4019 raft_consensus.cc:686] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 1 LEADER]: Becoming Leader. State: Replica: 29cecd30709e4b02b07cce67892c2678, State: Running, Role: LEADER
I20260806 16:11:13.480121 4019 consensus_queue.cc:237] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f31b4e268c04b8b9967d6130beb02fe" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34741 } } peers { permanent_uuid: "41a27cf20f824030a9e525b502ed4369" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46717 } } peers { permanent_uuid: "29cecd30709e4b02b07cce67892c2678" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41257 } }
I20260806 16:11:13.485019 3333 catalog_manager.cc:5495] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29cecd30709e4b02b07cce67892c2678 (127.8.49.1). New cstate: current_term: 1 leader_uuid: "29cecd30709e4b02b07cce67892c2678" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f31b4e268c04b8b9967d6130beb02fe" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41a27cf20f824030a9e525b502ed4369" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29cecd30709e4b02b07cce67892c2678" member_type: VOTER last_known_addr { host: "127.8.49.1" port: 41257 } health_report { overall_health: HEALTHY } } }
W20260806 16:11:13.489395 3985 tablet.cc:2369] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:11:13.502048 3785 tablet.cc:2369] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:11:13.510538 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.505478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37850 (local address 127.8.49.1:41257)
0806 16:11:13.505674 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:13.505678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:13.505692 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:11:13.505800 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:13.505802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:13.505861 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:13.505982 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:13.506525 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.507122 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.508152 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:13.508544 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:13.508760 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:13.509142 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:13.509943 (+ 801us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:13.510200 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:13.510272 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 16:11:13.510369 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 16:11:13.519423 3714 raft_consensus.cc:1261] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe [term 1 FOLLOWER]: Refusing update from remote peer 29cecd30709e4b02b07cce67892c2678: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:13.519423 3915 raft_consensus.cc:1261] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369 [term 1 FOLLOWER]: Refusing update from remote peer 29cecd30709e4b02b07cce67892c2678: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:13.520504 4019 consensus_queue.cc:1035] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f31b4e268c04b8b9967d6130beb02fe" member_type: VOTER last_known_addr { host: "127.8.49.2" port: 34741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:13.520989 4023 consensus_queue.cc:1035] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41a27cf20f824030a9e525b502ed4369" member_type: VOTER last_known_addr { host: "127.8.49.3" port: 46717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:13.531533 4027 mvcc.cc:205] Tried to move back new op lower bound from 7315589830722932736 to 7315589830579478528. Current Snapshot: MvccSnapshot[applied={T|T < 7315589830722932736}]
I20260806 16:11:13.535450 4028 mvcc.cc:205] Tried to move back new op lower bound from 7315589830722932736 to 7315589830579478528. Current Snapshot: MvccSnapshot[applied={T|T < 7315589830722932736}]
I20260806 16:11:13.548321 3333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:13.548580 3333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:11:13.552507 3333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8fb0fed6c16c48babf669f07cc391ccf: Sending DeleteTablet for 3 replicas of tablet cc23c49646424fe3933a72b1c4c086ea
I20260806 16:11:13.553934 3688 tablet_service.cc:1514] Processing DeleteTablet for tablet cc23c49646424fe3933a72b1c4c086ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:13 UTC) from {username='slave'} at 127.0.0.1:59080
I20260806 16:11:13.554037 3887 tablet_service.cc:1514] Processing DeleteTablet for tablet cc23c49646424fe3933a72b1c4c086ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:13 UTC) from {username='slave'} at 127.0.0.1:55022
I20260806 16:11:13.554306 3513 tablet_service.cc:1514] Processing DeleteTablet for tablet cc23c49646424fe3933a72b1c4c086ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:11:13 UTC) from {username='slave'} at 127.0.0.1:37846
I20260806 16:11:13.554883 4042 tablet_replica.cc:330] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: stopping tablet replica
I20260806 16:11:13.555088 4045 tablet_replica.cc:330] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678: stopping tablet replica
I20260806 16:11:13.555264 4044 tablet_replica.cc:330] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: stopping tablet replica
I20260806 16:11:13.555586 4045 raft_consensus.cc:2229] T cc23c49646424fe3933a72b1c4c086ea P 29cecd30709e4b02b07cce67892c2678 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:11:13.555606 4044 raft_consensus.cc:2229] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:11:13.555634 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 3432
I20260806 16:11:13.556416 4042 raft_consensus.cc:2229] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:11:13.556622 4044 raft_consensus.cc:2258] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:13.556996 4042 raft_consensus.cc:2258] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:11:13.562121 4044 ts_tablet_manager.cc:1905] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:13.568816 4044 ts_tablet_manager.cc:1918] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:13.569017 4044 log.cc:1196] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-2/wal/wals/cc23c49646424fe3933a72b1c4c086ea
I20260806 16:11:13.569859 4044 ts_tablet_manager.cc:1939] T cc23c49646424fe3933a72b1c4c086ea P 41a27cf20f824030a9e525b502ed4369: Deleting consensus metadata
I20260806 16:11:13.569883 4042 ts_tablet_manager.cc:1905] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:11:13.572050 3324 catalog_manager.cc:4841] TS 41a27cf20f824030a9e525b502ed4369 (127.8.49.3:46717): tablet cc23c49646424fe3933a72b1c4c086ea (table test-table [id=aa98e648b3ac43d38d710367a8562415]) successfully deleted
I20260806 16:11:13.574622 4042 ts_tablet_manager.cc:1918] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:11:13.574817 4042 log.cc:1196] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: Deleting WAL directory at /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786032364843570-8388-0/minicluster-data/ts-1/wal/wals/cc23c49646424fe3933a72b1c4c086ea
I20260806 16:11:13.575416 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 3611
I20260806 16:11:13.575518 4042 ts_tablet_manager.cc:1939] T cc23c49646424fe3933a72b1c4c086ea P 7f31b4e268c04b8b9967d6130beb02fe: Deleting consensus metadata
W20260806 16:11:13.575665 3324 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.49.1:41257 (error 108)
I20260806 16:11:13.585636 4005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.582408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.1:41257 (local address 127.0.0.1:37862)
0806 16:11:13.585322 (+ 2914us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:13.585481 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.1:41257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2725}
W20260806 16:11:13.585951 3324 catalog_manager.cc:4584] TS 29cecd30709e4b02b07cce67892c2678 (127.8.49.1:41257): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cc23c49646424fe3933a72b1c4c086ea: Network error: Client connection negotiation failed: client connection to 127.8.49.1:41257: connect: Connection refused (error 111)
I20260806 16:11:13.594434 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 3788
I20260806 16:11:13.594949 4005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:13.594563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.2:34741 (local address 127.0.0.1:59084)
0806 16:11:13.594743 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:13.594803 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.2:34741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 16:11:13.615202 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 3302
2026-08-06T16:11:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1496 ms)
[----------] 6 tests from EncryptionPolicyTest (14449 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:13 dist-test-slave-7z32 krb5kdc[4051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:13 dist-test-slave-7z32 krb5kdc[4051](info): set up 2 sockets
Aug 06 16:11:13 dist-test-slave-7z32 krb5kdc[4051](info): commencing operation
krb5kdc: starting...
W20260806 16:11:14.982057 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.002s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:15 dist-test-slave-7z32 krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:15.163506 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33487
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:15.340641 4068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:15.341308 4068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:15.341428 4068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:15.346475 4068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:15.346593 4068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:15.346617 4068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:15.346635 4068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:15.346658 4068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:15.353564 4068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33487
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:15.355299 4068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:15.356720 4068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:15.368364 4074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:15.368429 4075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:15.368374 4077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:15.368810 4068 server_base.cc:1034] running on GCE node
I20260806 16:11:15.369802 4068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:15.371480 4068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:15.372725 4068 hybrid_clock.cc:648] HybridClock initialized: now 1786032675372671 us; error 63 us; skew 500 ppm
Aug 06 16:11:15 dist-test-slave-7z32 krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032675, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:15.376948 4068 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:15.378499 4068 webserver.cc:460] Webserver started at http://127.0.0.1:42415/ using document root <none> and password file <none>
I20260806 16:11:15.379376 4068 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:15.379469 4068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:15.379707 4068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:15.382319 4068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "b64b768366594663beebb6a25332c13c"
format_stamp: "Formatted at 2026-08-06 16:11:15 on dist-test-slave-7z32"
server_key: "657d3fd16f302d5267471951de18a24c"
server_key_iv: "2df585bcc4626f021d9f3e8171ee009f"
server_key_version: "encryptionkey@0"
I20260806 16:11:15.383108 4068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "b64b768366594663beebb6a25332c13c"
format_stamp: "Formatted at 2026-08-06 16:11:15 on dist-test-slave-7z32"
server_key: "657d3fd16f302d5267471951de18a24c"
server_key_iv: "2df585bcc4626f021d9f3e8171ee009f"
server_key_version: "encryptionkey@0"
I20260806 16:11:15.388888 4068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:11:15.393944 4089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:15.396205 4068 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 16:11:15.396449 4068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "b64b768366594663beebb6a25332c13c"
format_stamp: "Formatted at 2026-08-06 16:11:15 on dist-test-slave-7z32"
server_key: "657d3fd16f302d5267471951de18a24c"
server_key_iv: "2df585bcc4626f021d9f3e8171ee009f"
server_key_version: "encryptionkey@0"
I20260806 16:11:15.396628 4068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:15.414546 4068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:15.419025 4068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:15.419304 4068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:15.435045 4068 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33487
I20260806 16:11:15.435163 4159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33487 every 8 connection(s)
I20260806 16:11:15.436667 4068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:15.437129 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 4068
I20260806 16:11:15.437323 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:15.437765 8388 external_mini_cluster.cc:1428] Setting key 4f5715fb451a07784d6d337bf4328866
I20260806 16:11:15.442030 4160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 16:11:15 dist-test-slave-7z32 krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:11:15.451692 4160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c: Bootstrap starting.
I20260806 16:11:15.455086 4160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:15.456107 4160 log.cc:824] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:15.458148 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:15.439971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46753 (local address 127.0.0.1:33487)
0806 16:11:15.440700 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:15.440721 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:15.441009 (+ 288us) server_negotiation.cc:407] Connection header received
0806 16:11:15.442249 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:15.442290 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:15.442776 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:15.443224 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:15.444465 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:15.445604 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:15.447066 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:15.447618 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:15.447769 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:15.451712 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:15.451719 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:15.451737 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:15.451777 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:15.454287 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:15.455115 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:15.455121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:15.455128 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:15.455305 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:15.455715 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:15.455721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:15.455728 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:15.456258 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:15.456549 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:15.456949 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 16:11:15.457233 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":235,"threads_started":1}
I20260806 16:11:15.460417 4160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c: No bootstrap required, opened a new log
I20260806 16:11:15.462751 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:15.466130 4160 raft_consensus.cc:348] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64b768366594663beebb6a25332c13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33487 } }
I20260806 16:11:15.466588 4160 raft_consensus.cc:374] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:15.466699 4160 raft_consensus.cc:729] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b64b768366594663beebb6a25332c13c, State: Initialized, Role: FOLLOWER
I20260806 16:11:15.467386 4160 consensus_queue.cc:260] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64b768366594663beebb6a25332c13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33487 } }
I20260806 16:11:15.467590 4160 raft_consensus.cc:388] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:15.467679 4160 raft_consensus.cc:482] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:15.467803 4160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:15.469800 4160 raft_consensus.cc:504] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64b768366594663beebb6a25332c13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33487 } }
I20260806 16:11:15.470614 4160 leader_election.cc:302] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b64b768366594663beebb6a25332c13c; no voters:
I20260806 16:11:15.471205 4160 leader_election.cc:288] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:15.471513 4173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:15.472054 4173 raft_consensus.cc:686] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [term 1 LEADER]: Becoming Leader. State: Replica: b64b768366594663beebb6a25332c13c, State: Running, Role: LEADER
I20260806 16:11:15.472625 4173 consensus_queue.cc:237] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64b768366594663beebb6a25332c13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33487 } }
I20260806 16:11:15.473640 4160 sys_catalog.cc:564] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:15.475201 4174 sys_catalog.cc:455] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b64b768366594663beebb6a25332c13c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64b768366594663beebb6a25332c13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33487 } } }
I20260806 16:11:15.475404 4174 sys_catalog.cc:458] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:15.475788 4175 sys_catalog.cc:455] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b64b768366594663beebb6a25332c13c. Latest consensus state: current_term: 1 leader_uuid: "b64b768366594663beebb6a25332c13c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b64b768366594663beebb6a25332c13c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33487 } } }
I20260806 16:11:15.475939 4175 sys_catalog.cc:458] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:15.476333 4186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:15.478055 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:15.465751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37695 (local address 127.0.0.1:33487)
0806 16:11:15.465963 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:15.465969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:15.465994 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:11:15.466049 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:15.466052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:15.466121 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:15.466259 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:15.467309 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:15.467991 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:15.469060 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:15.469353 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:15.469486 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:15.472075 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:15.472082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:15.472085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:15.472127 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:15.474712 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:15.475788 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:15.475793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:15.475796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:15.475918 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:15.476807 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:15.476812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:15.476813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:15.477068 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:15.477200 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:15.477762 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 16:11:15.477891 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 16:11:15.480759 4186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:15.489283 4186 catalog_manager.cc:1347] Generated new cluster ID: f6d33d0c664d431fb1695b9f9208a60e
I20260806 16:11:15.489400 4186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:15.502942 4186 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:15.505089 4186 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:15.515890 4186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b64b768366594663beebb6a25332c13c: Generated new TSK 0
I20260806 16:11:15.517239 4186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:15.523123 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 4068
W20260806 16:11:15.547130 4049 connection.cc:535] client connection to 127.0.0.1:33487 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33487): Connection reset by peer (error 104)
2026-08-06T16:11:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1912 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:15 dist-test-slave-7z32 krb5kdc[4203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:15 dist-test-slave-7z32 krb5kdc[4203](info): set up 2 sockets
Aug 06 16:11:15 dist-test-slave-7z32 krb5kdc[4203](info): commencing operation
krb5kdc: starting...
W20260806 16:11:16.913214 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.005s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:17 dist-test-slave-7z32 krb5kdc[4203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:17.095603 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44287
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:17.234534 4223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:17.234990 4223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:17.235049 4223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:17.239367 4223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:17.239460 4223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:17.239481 4223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:17.239499 4223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:17.239516 4223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:17.245512 4223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44287
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:17.246887 4223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:17.248114 4223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:17.259007 4236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:17.259033 4233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:17.259020 4232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:17.259364 4223 server_base.cc:1034] running on GCE node
I20260806 16:11:17.260241 4223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:17.261911 4223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:17.263183 4223 hybrid_clock.cc:648] HybridClock initialized: now 1786032677263134 us; error 64 us; skew 500 ppm
Aug 06 16:11:17 dist-test-slave-7z32 krb5kdc[4203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:17.271284 4223 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:17.272785 4223 webserver.cc:460] Webserver started at http://127.0.0.1:39313/ using document root <none> and password file <none>
I20260806 16:11:17.273617 4223 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:17.273736 4223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:17.274039 4223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:17.277277 4223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "b6580f2553794f3cbe9c34b01583e33e"
format_stamp: "Formatted at 2026-08-06 16:11:17 on dist-test-slave-7z32"
server_key: "58b38851787435457045e4aa19008d9d"
server_key_iv: "df8d34ccceee15db6f580a7aeb60e494"
server_key_version: "encryptionkey@0"
I20260806 16:11:17.278174 4223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "b6580f2553794f3cbe9c34b01583e33e"
format_stamp: "Formatted at 2026-08-06 16:11:17 on dist-test-slave-7z32"
server_key: "58b38851787435457045e4aa19008d9d"
server_key_iv: "df8d34ccceee15db6f580a7aeb60e494"
server_key_version: "encryptionkey@0"
I20260806 16:11:17.285625 4223 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 16:11:17.291363 4243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:17.293453 4223 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 16:11:17.293718 4223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "b6580f2553794f3cbe9c34b01583e33e"
format_stamp: "Formatted at 2026-08-06 16:11:17 on dist-test-slave-7z32"
server_key: "58b38851787435457045e4aa19008d9d"
server_key_iv: "df8d34ccceee15db6f580a7aeb60e494"
server_key_version: "encryptionkey@0"
I20260806 16:11:17.293905 4223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:17.306622 4223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:17.311625 4223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:17.311894 4223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:17.321275 4223 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44287
I20260806 16:11:17.321388 4314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44287 every 8 connection(s)
I20260806 16:11:17.322446 4223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:17.326530 4315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:17.327170 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 4223
I20260806 16:11:17.327337 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:17.327765 8388 external_mini_cluster.cc:1428] Setting key 7299a27b525e1f6f5a6fce80332aa7b7
I20260806 16:11:17.335762 4315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e: Bootstrap starting.
I20260806 16:11:17.338615 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:17.329598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45289 (local address 127.0.0.1:44287)
0806 16:11:17.330322 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:17.330343 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:17.330395 (+ 52us) server_negotiation.cc:407] Connection header received
0806 16:11:17.331422 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:17.331445 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:17.331880 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:17.332270 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:17.332742 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:17.333794 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:17.334758 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:17.335367 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:17.335572 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:17.336645 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:17.336650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:17.336667 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:17.337138 (+ 471us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:17.337206 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:17.337232 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:17.337389 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:17.337580 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 16:11:17.337746 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":210,"threads_started":1}
I20260806 16:11:17.339623 4315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:17.340716 4315 log.cc:824] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:17.343057 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:17.343973 4315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e: No bootstrap required, opened a new log
I20260806 16:11:17.349025 4315 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6580f2553794f3cbe9c34b01583e33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44287 } }
I20260806 16:11:17.349478 4315 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:17.349565 4315 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6580f2553794f3cbe9c34b01583e33e, State: Initialized, Role: FOLLOWER
I20260806 16:11:17.350203 4315 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6580f2553794f3cbe9c34b01583e33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44287 } }
I20260806 16:11:17.350392 4315 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:17.350492 4315 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:17.350636 4315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:17.350953 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:17.345558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58527 (local address 127.0.0.1:44287)
0806 16:11:17.345697 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:17.345701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:17.345835 (+ 134us) server_negotiation.cc:407] Connection header received
0806 16:11:17.346008 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:17.346011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:17.346069 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:17.346153 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:17.346538 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:17.347593 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:17.348860 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:17.349114 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:17.349201 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:17.350369 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:17.350373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:17.350376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:17.350393 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:17.350420 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:17.350435 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:17.350523 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:17.350715 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 16:11:17.350753 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 16:11:17.352085 4315 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6580f2553794f3cbe9c34b01583e33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44287 } }
I20260806 16:11:17.352550 4315 leader_election.cc:302] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6580f2553794f3cbe9c34b01583e33e; no voters:
I20260806 16:11:17.352996 4315 leader_election.cc:288] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:17.353257 4328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:17.353787 4328 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [term 1 LEADER]: Becoming Leader. State: Replica: b6580f2553794f3cbe9c34b01583e33e, State: Running, Role: LEADER
I20260806 16:11:17.354233 4328 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6580f2553794f3cbe9c34b01583e33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44287 } }
I20260806 16:11:17.354717 4315 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:17.356312 4331 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6580f2553794f3cbe9c34b01583e33e. Latest consensus state: current_term: 1 leader_uuid: "b6580f2553794f3cbe9c34b01583e33e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6580f2553794f3cbe9c34b01583e33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44287 } } }
I20260806 16:11:17.356455 4331 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:17.357026 4340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:17.356936 4329 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6580f2553794f3cbe9c34b01583e33e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6580f2553794f3cbe9c34b01583e33e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44287 } } }
I20260806 16:11:17.357126 4329 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:17.361312 4340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:17.369029 4340 catalog_manager.cc:1347] Generated new cluster ID: 9d68cdd90d45495ea5512c217b4463b1
I20260806 16:11:17.369163 4340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:17.420480 4340 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:17.421957 4340 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:17.431516 4340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6580f2553794f3cbe9c34b01583e33e: Generated new TSK 0
I20260806 16:11:17.432523 4340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:17.441195 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 4223
W20260806 16:11:17.455631 4200 connection.cc:535] client connection to 127.0.0.1:44287 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44287): Connection reset by peer (error 104)
2026-08-06T16:11:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1907 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:17 dist-test-slave-7z32 krb5kdc[4356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:17 dist-test-slave-7z32 krb5kdc[4356](info): set up 2 sockets
Aug 06 16:11:17 dist-test-slave-7z32 krb5kdc[4356](info): commencing operation
krb5kdc: starting...
W20260806 16:11:18.800509 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.330s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:18 dist-test-slave-7z32 krb5kdc[4356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:18.993016 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:19.152654 4372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:19.153246 4372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:19.153401 4372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:19.158346 4372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:19.158479 4372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:19.158501 4372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:19.158521 4372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:19.158538 4372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:19.166338 4372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41709
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:19.167985 4372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:19.169327 4372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:19.183368 4382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:19.183398 4387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:19.183560 4385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:19.184140 4372 server_base.cc:1034] running on GCE node
I20260806 16:11:19.185153 4372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:19.187480 4372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:19.188901 4372 hybrid_clock.cc:648] HybridClock initialized: now 1786032679188871 us; error 51 us; skew 500 ppm
Aug 06 16:11:19 dist-test-slave-7z32 krb5kdc[4356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:19.194202 4372 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:19.195899 4372 webserver.cc:460] Webserver started at http://127.0.0.1:46107/ using document root <none> and password file <none>
I20260806 16:11:19.197155 4372 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:19.197283 4372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:19.197535 4372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:19.201930 4372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "5cbb30f8ad7e43258d48ec68fda74a92"
format_stamp: "Formatted at 2026-08-06 16:11:19 on dist-test-slave-7z32"
server_key: "216d6fed205b0f915012ad9b8c72b000"
server_key_iv: "27e713f92b4154f6947728ef6cb5fd7c"
server_key_version: "encryptionkey@0"
I20260806 16:11:19.202798 4372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "5cbb30f8ad7e43258d48ec68fda74a92"
format_stamp: "Formatted at 2026-08-06 16:11:19 on dist-test-slave-7z32"
server_key: "216d6fed205b0f915012ad9b8c72b000"
server_key_iv: "27e713f92b4154f6947728ef6cb5fd7c"
server_key_version: "encryptionkey@0"
I20260806 16:11:19.209470 4372 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 16:11:19.214061 4397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:19.216480 4372 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 16:11:19.216709 4372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "5cbb30f8ad7e43258d48ec68fda74a92"
format_stamp: "Formatted at 2026-08-06 16:11:19 on dist-test-slave-7z32"
server_key: "216d6fed205b0f915012ad9b8c72b000"
server_key_iv: "27e713f92b4154f6947728ef6cb5fd7c"
server_key_version: "encryptionkey@0"
I20260806 16:11:19.216962 4372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:19.223606 4372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:19.228462 4372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:19.228772 4372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:19.239909 4372 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41709
I20260806 16:11:19.239960 4464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41709 every 8 connection(s)
I20260806 16:11:19.241325 4372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:19.245137 4465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:19.246142 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 4372
I20260806 16:11:19.246296 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:19.246721 8388 external_mini_cluster.cc:1428] Setting key 0b4745c70a7125bb7a3887b1a6589a2a
I20260806 16:11:19.253221 4465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92: Bootstrap starting.
I20260806 16:11:19.255148 4468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:19.249045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42079 (local address 127.0.0.1:41709)
0806 16:11:19.249739 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:19.249765 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:19.249827 (+ 62us) server_negotiation.cc:407] Connection header received
0806 16:11:19.250748 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:19.250778 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:19.251232 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:19.251677 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:19.252829 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:19.252836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:19.252853 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:19.252868 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:19.252944 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:19.252974 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:19.253169 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:19.253684 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 16:11:19.253964 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":211,"threads_started":1}
I20260806 16:11:19.256312 4465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:19.257190 4465 log.cc:824] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:19.259603 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:19.260421 4465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92: No bootstrap required, opened a new log
I20260806 16:11:19.264894 4468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:19.262359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59651 (local address 127.0.0.1:41709)
0806 16:11:19.262537 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:19.262543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:19.262897 (+ 354us) server_negotiation.cc:407] Connection header received
0806 16:11:19.262990 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:19.262993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:19.263050 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:19.263271 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:19.264290 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:19.264293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:19.264295 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:19.264310 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:19.264345 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:19.264357 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:19.264462 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:19.264613 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 16:11:19.264666 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:11:19.265547 4465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cbb30f8ad7e43258d48ec68fda74a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } }
I20260806 16:11:19.265868 4465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:19.265951 4465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbb30f8ad7e43258d48ec68fda74a92, State: Initialized, Role: FOLLOWER
I20260806 16:11:19.266434 4465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cbb30f8ad7e43258d48ec68fda74a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } }
I20260806 16:11:19.266631 4465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:19.266713 4465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:19.266880 4465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:19.267979 4465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cbb30f8ad7e43258d48ec68fda74a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } }
I20260806 16:11:19.268613 4465 leader_election.cc:302] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cbb30f8ad7e43258d48ec68fda74a92; no voters:
I20260806 16:11:19.269073 4465 leader_election.cc:288] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:19.269364 4479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:19.270108 4479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [term 1 LEADER]: Becoming Leader. State: Replica: 5cbb30f8ad7e43258d48ec68fda74a92, State: Running, Role: LEADER
I20260806 16:11:19.270691 4479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cbb30f8ad7e43258d48ec68fda74a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } }
I20260806 16:11:19.270738 4465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:19.273164 4481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cbb30f8ad7e43258d48ec68fda74a92. Latest consensus state: current_term: 1 leader_uuid: "5cbb30f8ad7e43258d48ec68fda74a92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cbb30f8ad7e43258d48ec68fda74a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } } }
I20260806 16:11:19.273214 4480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cbb30f8ad7e43258d48ec68fda74a92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cbb30f8ad7e43258d48ec68fda74a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41709 } } }
I20260806 16:11:19.273865 4480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:19.273393 4481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92 [sys.catalog]: This master's current role is: LEADER
W20260806 16:11:19.276933 4498 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:11:19.277074 4498 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:19.277181 4499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:19.277976 4499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:19.286309 4499 catalog_manager.cc:1347] Generated new cluster ID: 7c7497e8d6164fcb98431e3b39b98e7c
I20260806 16:11:19.286423 4499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:19.311784 4499 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:19.312456 4499 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:19.321462 4499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cbb30f8ad7e43258d48ec68fda74a92: Generated new TSK 0
I20260806 16:11:19.322860 4499 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:19.330015 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 4372
2026-08-06T16:11:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1894 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:19 dist-test-slave-7z32 krb5kdc[4509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:19 dist-test-slave-7z32 krb5kdc[4509](info): set up 2 sockets
Aug 06 16:11:19 dist-test-slave-7z32 krb5kdc[4509](info): commencing operation
krb5kdc: starting...
W20260806 16:11:20.682087 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:20 dist-test-slave-7z32 krb5kdc[4509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:20.865046 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:21.030398 4525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:21.030869 4525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:21.030923 4525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:21.036576 4525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:21.036739 4525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:21.036767 4525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:21.036789 4525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:21.036814 4525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:21.043876 4525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:21.045467 4525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:21.046870 4525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:21.059727 4534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:21.059727 4532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:21.059810 4525 server_base.cc:1034] running on GCE node
W20260806 16:11:21.059727 4531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:21.060880 4525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:21.062352 4525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:21.063611 4525 hybrid_clock.cc:648] HybridClock initialized: now 1786032681063538 us; error 92 us; skew 500 ppm
Aug 06 16:11:21 dist-test-slave-7z32 krb5kdc[4509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:21.067559 4525 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:21.068696 4525 webserver.cc:460] Webserver started at http://127.0.0.1:40251/ using document root <none> and password file <none>
I20260806 16:11:21.069398 4525 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:21.069483 4525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:21.069731 4525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:21.072178 4525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "fed05826ab2247e29a6a55f1c4589ee0"
format_stamp: "Formatted at 2026-08-06 16:11:21 on dist-test-slave-7z32"
server_key: "8e336a56f6c20d32bef71d76a2dbcfb0"
server_key_iv: "010a605b2e5f60ded3c37e31d01129b1"
server_key_version: "encryptionkey@0"
I20260806 16:11:21.072832 4525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "fed05826ab2247e29a6a55f1c4589ee0"
format_stamp: "Formatted at 2026-08-06 16:11:21 on dist-test-slave-7z32"
server_key: "8e336a56f6c20d32bef71d76a2dbcfb0"
server_key_iv: "010a605b2e5f60ded3c37e31d01129b1"
server_key_version: "encryptionkey@0"
I20260806 16:11:21.077854 4525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:11:21.081310 4543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:21.083298 4525 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:21.083506 4525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "fed05826ab2247e29a6a55f1c4589ee0"
format_stamp: "Formatted at 2026-08-06 16:11:21 on dist-test-slave-7z32"
server_key: "8e336a56f6c20d32bef71d76a2dbcfb0"
server_key_iv: "010a605b2e5f60ded3c37e31d01129b1"
server_key_version: "encryptionkey@0"
I20260806 16:11:21.083668 4525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:21.119364 4525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:21.123210 4525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:21.123481 4525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:21.133349 4525 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42977
I20260806 16:11:21.133438 4616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42977 every 8 connection(s)
I20260806 16:11:21.134701 4525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:21.137439 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 4525
I20260806 16:11:21.137634 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:21.138010 8388 external_mini_cluster.cc:1428] Setting key a419407cdce8271894dd375c88f1e59a
I20260806 16:11:21.139217 4617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:21.148617 4617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0: Bootstrap starting.
Aug 06 16:11:21 dist-test-slave-7z32 krb5kdc[4509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:11:21.152629 4617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:21.153891 4617 log.cc:824] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:21.157534 4617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0: No bootstrap required, opened a new log
I20260806 16:11:21.159118 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:21.140024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50697 (local address 127.0.0.1:42977)
0806 16:11:21.140616 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:21.140643 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:21.140735 (+ 92us) server_negotiation.cc:407] Connection header received
0806 16:11:21.141825 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:21.141856 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:21.142453 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:21.143020 (+ 567us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:21.144196 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:21.145833 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:21.147059 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:21.147830 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:21.148030 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:21.151854 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:21.151863 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:21.151886 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:21.151937 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:21.154541 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:21.155428 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:21.155435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:21.155443 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:21.155617 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:21.156146 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:21.156149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:21.156150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:21.156815 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:21.157036 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:21.157442 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 16:11:21.157862 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":148,"threads_started":1}
I20260806 16:11:21.163019 4617 raft_consensus.cc:348] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed05826ab2247e29a6a55f1c4589ee0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } }
I20260806 16:11:21.163363 4617 raft_consensus.cc:374] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:21.163421 4617 raft_consensus.cc:729] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed05826ab2247e29a6a55f1c4589ee0, State: Initialized, Role: FOLLOWER
I20260806 16:11:21.163926 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:21.164184 4617 consensus_queue.cc:260] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed05826ab2247e29a6a55f1c4589ee0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } }
I20260806 16:11:21.164384 4617 raft_consensus.cc:388] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:21.164467 4617 raft_consensus.cc:482] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:21.164639 4617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:21.166044 4617 raft_consensus.cc:504] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed05826ab2247e29a6a55f1c4589ee0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } }
I20260806 16:11:21.166534 4617 leader_election.cc:302] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fed05826ab2247e29a6a55f1c4589ee0; no voters:
I20260806 16:11:21.166994 4617 leader_election.cc:288] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:21.167498 4632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:21.167821 4632 raft_consensus.cc:686] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [term 1 LEADER]: Becoming Leader. State: Replica: fed05826ab2247e29a6a55f1c4589ee0, State: Running, Role: LEADER
I20260806 16:11:21.168300 4632 consensus_queue.cc:237] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed05826ab2247e29a6a55f1c4589ee0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } }
I20260806 16:11:21.168869 4617 sys_catalog.cc:564] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:21.169919 4632 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fed05826ab2247e29a6a55f1c4589ee0. Latest consensus state: current_term: 1 leader_uuid: "fed05826ab2247e29a6a55f1c4589ee0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed05826ab2247e29a6a55f1c4589ee0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } } }
I20260806 16:11:21.170171 4632 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:21.170423 4633 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fed05826ab2247e29a6a55f1c4589ee0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed05826ab2247e29a6a55f1c4589ee0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42977 } } }
I20260806 16:11:21.170558 4633 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:21.170814 4644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:21.174996 4644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:21.177894 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:21.166790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55387 (local address 127.0.0.1:42977)
0806 16:11:21.167033 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:21.167040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:21.167084 (+ 44us) server_negotiation.cc:407] Connection header received
0806 16:11:21.167262 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:21.167265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:21.167357 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:21.167450 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:21.168504 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:21.169322 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:21.170675 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:21.171144 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:21.171246 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:21.173628 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:21.173633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:21.173637 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:21.173671 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:21.175644 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:21.176215 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:21.176219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:21.176221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:21.176319 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:21.176624 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:21.176628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:21.176630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:21.176864 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:21.177027 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:21.177529 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 16:11:21.177686 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 16:11:21.184751 4644 catalog_manager.cc:1347] Generated new cluster ID: f0f710313113429c959fbdadde795071
I20260806 16:11:21.184880 4644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:21.206893 4644 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:21.208472 4644 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:21.216323 4644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fed05826ab2247e29a6a55f1c4589ee0: Generated new TSK 0
I20260806 16:11:21.217398 4644 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:21.229132 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 4525
2026-08-06T16:11:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1896 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:21 dist-test-slave-7z32 krb5kdc[4657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:21 dist-test-slave-7z32 krb5kdc[4657](info): set up 2 sockets
Aug 06 16:11:21 dist-test-slave-7z32 krb5kdc[4657](info): commencing operation
krb5kdc: starting...
W20260806 16:11:22.595660 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:22 dist-test-slave-7z32 krb5kdc[4657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:22.779441 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42363
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:22.926474 4674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:22.926999 4674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:22.927063 4674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:22.932838 4674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:22.932971 4674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:22.932993 4674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:22.933013 4674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:22.933032 4674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:22.940917 4674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:22.942795 4674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:22.944380 4674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:11:22.956841 4674 server_base.cc:1034] running on GCE node
W20260806 16:11:22.956758 4684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:22.956779 4683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:22.956779 4686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:22.958489 4674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:22.960292 4674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:22.961548 4674 hybrid_clock.cc:648] HybridClock initialized: now 1786032682961490 us; error 82 us; skew 500 ppm
Aug 06 16:11:22 dist-test-slave-7z32 krb5kdc[4657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:22.973048 4674 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:22.974776 4674 webserver.cc:460] Webserver started at http://127.0.0.1:34063/ using document root <none> and password file <none>
I20260806 16:11:22.975697 4674 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:22.975813 4674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:22.976121 4674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:22.979171 4674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "8657c71932054a90a2d4cdf356fc49c5"
format_stamp: "Formatted at 2026-08-06 16:11:22 on dist-test-slave-7z32"
server_key: "0d736357a696b42ce41052833c2f1072"
server_key_iv: "86d8970cb3687545dd7cfc2bfc9c55c2"
server_key_version: "encryptionkey@0"
I20260806 16:11:22.979983 4674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "8657c71932054a90a2d4cdf356fc49c5"
format_stamp: "Formatted at 2026-08-06 16:11:22 on dist-test-slave-7z32"
server_key: "0d736357a696b42ce41052833c2f1072"
server_key_iv: "86d8970cb3687545dd7cfc2bfc9c55c2"
server_key_version: "encryptionkey@0"
I20260806 16:11:22.985977 4674 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260806 16:11:22.990543 4693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:22.992792 4674 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 16:11:22.993046 4674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "8657c71932054a90a2d4cdf356fc49c5"
format_stamp: "Formatted at 2026-08-06 16:11:22 on dist-test-slave-7z32"
server_key: "0d736357a696b42ce41052833c2f1072"
server_key_iv: "86d8970cb3687545dd7cfc2bfc9c55c2"
server_key_version: "encryptionkey@0"
I20260806 16:11:22.993397 4674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:23.013202 4674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:23.018193 4674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:23.018527 4674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:23.029557 4674 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42363
I20260806 16:11:23.029726 4766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42363 every 8 connection(s)
I20260806 16:11:23.031185 4674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:23.031781 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 4674
I20260806 16:11:23.031975 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:23.032371 8388 external_mini_cluster.cc:1428] Setting key 2759497d8cbc9e06ce3a78a916053a58
I20260806 16:11:23.036055 4767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:23.045369 4767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5: Bootstrap starting.
I20260806 16:11:23.045915 4770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:23.034466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40715 (local address 127.0.0.1:42363)
0806 16:11:23.035113 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:23.035146 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:23.035197 (+ 51us) server_negotiation.cc:407] Connection header received
0806 16:11:23.036080 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:23.036103 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:23.036658 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:23.037070 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:23.037701 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:23.039089 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:23.040766 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:23.041420 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:23.041591 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:23.043028 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:23.043035 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:23.043058 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:23.043730 (+ 672us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:23.043820 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:23.043852 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:23.043969 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:23.044322 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 16:11:23.044759 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":286,"threads_started":1}
I20260806 16:11:23.049073 4767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:23.050441 4767 log.cc:824] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:23.050508 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:23.053699 4767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5: No bootstrap required, opened a new log
I20260806 16:11:23.058902 4767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8657c71932054a90a2d4cdf356fc49c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42363 } }
I20260806 16:11:23.059258 4767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:23.059297 4770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:23.053267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43617 (local address 127.0.0.1:42363)
0806 16:11:23.053465 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:23.053471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:23.053824 (+ 353us) server_negotiation.cc:407] Connection header received
0806 16:11:23.053983 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:23.053986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:23.054050 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:23.054141 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:23.054582 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:23.055582 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:23.056976 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:23.057254 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:23.057366 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:23.058504 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:23.058507 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:23.058510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:23.058528 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:23.058565 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:23.058577 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:23.058657 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:23.059005 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 16:11:23.059114 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 16:11:23.059345 4767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8657c71932054a90a2d4cdf356fc49c5, State: Initialized, Role: FOLLOWER
I20260806 16:11:23.060137 4767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8657c71932054a90a2d4cdf356fc49c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42363 } }
I20260806 16:11:23.060328 4767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:23.060392 4767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:23.060509 4767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:23.061779 4767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8657c71932054a90a2d4cdf356fc49c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42363 } }
I20260806 16:11:23.062151 4767 leader_election.cc:302] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8657c71932054a90a2d4cdf356fc49c5; no voters:
I20260806 16:11:23.062490 4767 leader_election.cc:288] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:23.062887 4781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:23.063244 4781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [term 1 LEADER]: Becoming Leader. State: Replica: 8657c71932054a90a2d4cdf356fc49c5, State: Running, Role: LEADER
I20260806 16:11:23.063704 4781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8657c71932054a90a2d4cdf356fc49c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42363 } }
I20260806 16:11:23.064173 4767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:23.065935 4782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8657c71932054a90a2d4cdf356fc49c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8657c71932054a90a2d4cdf356fc49c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42363 } } }
I20260806 16:11:23.066143 4782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:23.066541 4783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8657c71932054a90a2d4cdf356fc49c5. Latest consensus state: current_term: 1 leader_uuid: "8657c71932054a90a2d4cdf356fc49c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8657c71932054a90a2d4cdf356fc49c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42363 } } }
I20260806 16:11:23.066643 4783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:23.066992 4793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:23.070963 4793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:23.077499 4793 catalog_manager.cc:1347] Generated new cluster ID: 48e914e486d84f8aa762d7a9480b7b10
I20260806 16:11:23.077630 4793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:23.112720 4793 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:23.114209 4793 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:23.129040 4793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8657c71932054a90a2d4cdf356fc49c5: Generated new TSK 0
I20260806 16:11:23.130421 4793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:23.147789 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 4674
2026-08-06T16:11:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1923 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:23 dist-test-slave-7z32 krb5kdc[4807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:23 dist-test-slave-7z32 krb5kdc[4807](info): set up 2 sockets
Aug 06 16:11:23 dist-test-slave-7z32 krb5kdc[4807](info): commencing operation
krb5kdc: starting...
W20260806 16:11:24.572386 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.386s user 0.006s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:24 dist-test-slave-7z32 krb5kdc[4807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:24.764359 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44795
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:24.931982 4824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:24.932464 4824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:24.932564 4824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:24.938913 4824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:24.939086 4824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:24.939129 4824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:24.939177 4824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:24.939239 4824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:24.946429 4824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:24.948278 4824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:24.949870 4824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:24.962235 4830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:24.962218 4835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:24.962221 4833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:24.962302 4824 server_base.cc:1034] running on GCE node
I20260806 16:11:24.963533 4824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:24.965252 4824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:24.966578 4824 hybrid_clock.cc:648] HybridClock initialized: now 1786032684966532 us; error 67 us; skew 500 ppm
Aug 06 16:11:24 dist-test-slave-7z32 krb5kdc[4807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032684, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:24.970916 4824 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:24.972362 4824 webserver.cc:460] Webserver started at http://127.0.0.1:36131/ using document root <none> and password file <none>
I20260806 16:11:24.973208 4824 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:24.973327 4824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:24.973639 4824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:24.977411 4824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "01001626ba5e46648f77917ce9408892"
format_stamp: "Formatted at 2026-08-06 16:11:24 on dist-test-slave-7z32"
server_key: "eb1a2df6331f63c5344b2a659ef5bb12"
server_key_iv: "ac03a4f1e2e2c974648dc967a8bb64c1"
server_key_version: "encryptionkey@0"
I20260806 16:11:24.978264 4824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "01001626ba5e46648f77917ce9408892"
format_stamp: "Formatted at 2026-08-06 16:11:24 on dist-test-slave-7z32"
server_key: "eb1a2df6331f63c5344b2a659ef5bb12"
server_key_iv: "ac03a4f1e2e2c974648dc967a8bb64c1"
server_key_version: "encryptionkey@0"
I20260806 16:11:24.985911 4824 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260806 16:11:24.991588 4844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:24.993527 4824 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260806 16:11:24.993856 4824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "01001626ba5e46648f77917ce9408892"
format_stamp: "Formatted at 2026-08-06 16:11:24 on dist-test-slave-7z32"
server_key: "eb1a2df6331f63c5344b2a659ef5bb12"
server_key_iv: "ac03a4f1e2e2c974648dc967a8bb64c1"
server_key_version: "encryptionkey@0"
I20260806 16:11:24.994079 4824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:25.001077 4824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:25.006069 4824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:25.006389 4824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:25.016899 4824 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44795
I20260806 16:11:25.017045 4916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44795 every 8 connection(s)
I20260806 16:11:25.018417 4824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:25.022221 4917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:25.027212 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 4824
I20260806 16:11:25.027410 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:25.027725 8388 external_mini_cluster.cc:1428] Setting key c13007dc193549ef1e61004fb4df9138
I20260806 16:11:25.029702 4917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892: Bootstrap starting.
I20260806 16:11:25.032565 4917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:25.033751 4917 log.cc:824] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:25.036229 4921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:25.029864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54857 (local address 127.0.0.1:44795)
0806 16:11:25.030583 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:25.030604 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:25.030664 (+ 60us) server_negotiation.cc:407] Connection header received
0806 16:11:25.031675 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:25.031701 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:25.032176 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:25.032590 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:25.033855 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:25.033862 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:25.033884 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:25.033975 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:25.034099 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:25.034143 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:25.034312 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:25.034416 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 16:11:25.034955 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":263,"threads_started":1}
I20260806 16:11:25.036845 4917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892: No bootstrap required, opened a new log
I20260806 16:11:25.040773 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:25.042194 4917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01001626ba5e46648f77917ce9408892" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44795 } }
I20260806 16:11:25.042573 4917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:25.042706 4917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01001626ba5e46648f77917ce9408892, State: Initialized, Role: FOLLOWER
I20260806 16:11:25.043354 4917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01001626ba5e46648f77917ce9408892" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44795 } }
I20260806 16:11:25.043536 4917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:25.043587 4917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:25.043718 4917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:25.045192 4917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01001626ba5e46648f77917ce9408892" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44795 } }
I20260806 16:11:25.045595 4917 leader_election.cc:302] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01001626ba5e46648f77917ce9408892; no voters:
I20260806 16:11:25.046017 4917 leader_election.cc:288] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:25.046211 4930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:25.046711 4921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:25.044070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34291 (local address 127.0.0.1:44795)
0806 16:11:25.044325 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:25.044330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:25.044570 (+ 240us) server_negotiation.cc:407] Connection header received
0806 16:11:25.044731 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:25.044735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:25.044791 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:25.044915 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:25.045966 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:25.045970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:25.045973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:25.045987 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:25.046024 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:25.046042 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:25.046153 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:25.046434 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 16:11:25.046504 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260806 16:11:25.046782 4930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [term 1 LEADER]: Becoming Leader. State: Replica: 01001626ba5e46648f77917ce9408892, State: Running, Role: LEADER
I20260806 16:11:25.047323 4930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01001626ba5e46648f77917ce9408892" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44795 } }
I20260806 16:11:25.047544 4917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:25.049755 4932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01001626ba5e46648f77917ce9408892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01001626ba5e46648f77917ce9408892" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44795 } } }
I20260806 16:11:25.049798 4931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01001626ba5e46648f77917ce9408892. Latest consensus state: current_term: 1 leader_uuid: "01001626ba5e46648f77917ce9408892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01001626ba5e46648f77917ce9408892" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44795 } } }
I20260806 16:11:25.049934 4932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:25.049948 4931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892 [sys.catalog]: This master's current role is: LEADER
W20260806 16:11:25.053895 4948 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:11:25.054009 4948 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:25.054088 4949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:25.055044 4949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:25.061996 4949 catalog_manager.cc:1347] Generated new cluster ID: fa22e883b52f460899af48cb79d64a18
I20260806 16:11:25.062122 4949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:25.076587 4949 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:25.077087 4949 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:25.086841 4949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01001626ba5e46648f77917ce9408892: Generated new TSK 0
I20260806 16:11:25.087972 4949 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:25.094712 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 4824
2026-08-06T16:11:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1947 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:25 dist-test-slave-7z32 krb5kdc[4959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:25 dist-test-slave-7z32 krb5kdc[4959](info): set up 2 sockets
Aug 06 16:11:25 dist-test-slave-7z32 krb5kdc[4959](info): commencing operation
krb5kdc: starting...
W20260806 16:11:26.497434 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:26 dist-test-slave-7z32 krb5kdc[4959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:26.689477 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38849
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:26.858507 4976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:26.859097 4976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:26.859195 4976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:26.864157 4976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:26.864444 4976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:26.864526 4976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:26.864565 4976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:26.864609 4976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:26.871865 4976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:26.874039 4976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:26.875976 4976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:26.889698 4984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:26.890007 4976 server_base.cc:1034] running on GCE node
W20260806 16:11:26.889720 4986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:26.889770 4983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:26.891106 4976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:26.892670 4976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:26.893927 4976 hybrid_clock.cc:648] HybridClock initialized: now 1786032686893887 us; error 70 us; skew 500 ppm
Aug 06 16:11:26 dist-test-slave-7z32 krb5kdc[4959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032686, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:26.898058 4976 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:26.899501 4976 webserver.cc:460] Webserver started at http://127.0.0.1:46053/ using document root <none> and password file <none>
I20260806 16:11:26.900367 4976 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:26.900484 4976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:26.900785 4976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:26.903633 4976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "351d7113c28f439e806fcfd0ba9f1990"
format_stamp: "Formatted at 2026-08-06 16:11:26 on dist-test-slave-7z32"
server_key: "a441456a0eb19f762b41030ea071686f"
server_key_iv: "9c6255bb44097fa0ff2538981db7f2fd"
server_key_version: "encryptionkey@0"
I20260806 16:11:26.904379 4976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "351d7113c28f439e806fcfd0ba9f1990"
format_stamp: "Formatted at 2026-08-06 16:11:26 on dist-test-slave-7z32"
server_key: "a441456a0eb19f762b41030ea071686f"
server_key_iv: "9c6255bb44097fa0ff2538981db7f2fd"
server_key_version: "encryptionkey@0"
I20260806 16:11:26.909962 4976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:11:26.913894 4999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:26.915917 4976 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:11:26.916136 4976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "351d7113c28f439e806fcfd0ba9f1990"
format_stamp: "Formatted at 2026-08-06 16:11:26 on dist-test-slave-7z32"
server_key: "a441456a0eb19f762b41030ea071686f"
server_key_iv: "9c6255bb44097fa0ff2538981db7f2fd"
server_key_version: "encryptionkey@0"
I20260806 16:11:26.916302 4976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:26.927120 4976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:26.931324 4976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:26.931619 4976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:26.941063 4976 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38849
I20260806 16:11:26.941149 5071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38849 every 8 connection(s)
I20260806 16:11:26.942226 4976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:26.947139 5076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:26.952401 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 4976
I20260806 16:11:26.952600 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:26.952942 8388 external_mini_cluster.cc:1428] Setting key 8e6b6f40249bb55c016b29248a5b4245
I20260806 16:11:26.956413 5076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990: Bootstrap starting.
I20260806 16:11:26.959646 5076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:26.961172 5076 log.cc:824] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:26.964488 5076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990: No bootstrap required, opened a new log
Aug 06 16:11:26 dist-test-slave-7z32 krb5kdc[4959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:11:26.971504 5076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351d7113c28f439e806fcfd0ba9f1990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38849 } }
I20260806 16:11:26.971885 5076 raft_consensus.cc:374] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:26.971930 5076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351d7113c28f439e806fcfd0ba9f1990, State: Initialized, Role: FOLLOWER
I20260806 16:11:26.972692 5076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351d7113c28f439e806fcfd0ba9f1990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38849 } }
I20260806 16:11:26.972900 5076 raft_consensus.cc:388] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:26.973016 5076 raft_consensus.cc:482] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:26.973201 5076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:26.975013 5076 raft_consensus.cc:504] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351d7113c28f439e806fcfd0ba9f1990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38849 } }
I20260806 16:11:26.975525 5076 leader_election.cc:302] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 351d7113c28f439e806fcfd0ba9f1990; no voters:
I20260806 16:11:26.975997 5076 leader_election.cc:288] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:26.976308 5083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:26.976688 5079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:26.955331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:54535 (local address 127.0.0.1:38849)
0806 16:11:26.956254 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:26.956312 (+ 58us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:26.956371 (+ 59us) server_negotiation.cc:407] Connection header received
0806 16:11:26.957483 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:26.957515 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:26.957967 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:26.958421 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:26.959373 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:26.960520 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:26.961519 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:26.962422 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:26.962708 (+ 286us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:26.966997 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:26.967006 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:26.967038 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:26.967092 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:26.971198 (+ 4106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:26.972373 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:26.972382 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:26.972390 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:26.972594 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:26.973127 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:26.973140 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:26.973142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:26.973856 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:26.974154 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:26.974614 (+ 460us) server_negotiation.cc:299] Negotiation successful
0806 16:11:26.974993 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":214,"threads_started":1}
I20260806 16:11:26.976790 5083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [term 1 LEADER]: Becoming Leader. State: Replica: 351d7113c28f439e806fcfd0ba9f1990, State: Running, Role: LEADER
I20260806 16:11:26.977597 5083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351d7113c28f439e806fcfd0ba9f1990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38849 } }
I20260806 16:11:26.978179 5076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:26.980805 5086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "351d7113c28f439e806fcfd0ba9f1990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351d7113c28f439e806fcfd0ba9f1990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38849 } } }
I20260806 16:11:26.981026 5086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:26.981487 5085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 351d7113c28f439e806fcfd0ba9f1990. Latest consensus state: current_term: 1 leader_uuid: "351d7113c28f439e806fcfd0ba9f1990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "351d7113c28f439e806fcfd0ba9f1990" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38849 } } }
I20260806 16:11:26.982542 5085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:26.983355 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 16:11:26.985915 5104 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:11:26.986039 5104 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:26.986140 5105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:26.987232 5105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:26.997068 5105 catalog_manager.cc:1347] Generated new cluster ID: 76516fcfa4f94cc89e83b169b6272b23
I20260806 16:11:26.997315 5105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:27.001598 5079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:26.989457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:54285 (local address 127.0.0.1:38849)
0806 16:11:26.989699 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:26.989706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:26.989732 (+ 26us) server_negotiation.cc:407] Connection header received
0806 16:11:26.989807 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:26.989812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:26.989895 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:26.990047 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:26.991240 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:26.992812 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:26.994003 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:26.994429 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:26.994581 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:26.997168 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:26.997173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:26.997176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:26.997226 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:26.999280 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:26.999971 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:26.999975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:26.999976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:27.000068 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:27.000490 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:27.000493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:27.000494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:27.000724 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:27.000953 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:27.001260 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 16:11:27.001386 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 16:11:27.012183 5105 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:27.014642 5105 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:27.022890 5105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 351d7113c28f439e806fcfd0ba9f1990: Generated new TSK 0
I20260806 16:11:27.023993 5105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:27.029397 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 4976
W20260806 16:11:27.043494 4957 connection.cc:535] client connection to 127.0.0.1:38849 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38849): Connection reset by peer (error 104)
2026-08-06T16:11:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1925 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:27 dist-test-slave-7z32 krb5kdc[5119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:27 dist-test-slave-7z32 krb5kdc[5119](info): set up 2 sockets
Aug 06 16:11:27 dist-test-slave-7z32 krb5kdc[5119](info): commencing operation
krb5kdc: starting...
W20260806 16:11:28.339205 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:28 dist-test-slave-7z32 krb5kdc[5119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:28.512854 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34895
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:28.674078 5141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:28.674602 5141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:28.674713 5141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:28.680032 5141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:28.680208 5141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:28.680246 5141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:28.680281 5141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:28.680320 5141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:28.686997 5141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:28.688987 5141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:28.690579 5141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:28.703063 5153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:28.703051 5149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:28.703049 5150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:28.703648 5141 server_base.cc:1034] running on GCE node
I20260806 16:11:28.704792 5141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:28.706247 5141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:28.707612 5141 hybrid_clock.cc:648] HybridClock initialized: now 1786032688707580 us; error 62 us; skew 500 ppm
Aug 06 16:11:28 dist-test-slave-7z32 krb5kdc[5119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032688, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:28.718797 5141 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:28.720331 5141 webserver.cc:460] Webserver started at http://127.0.0.1:37933/ using document root <none> and password file <none>
I20260806 16:11:28.721179 5141 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:28.721314 5141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:28.721621 5141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:28.724910 5141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "f19870b4111348e1aa16136d83ab01b2"
format_stamp: "Formatted at 2026-08-06 16:11:28 on dist-test-slave-7z32"
server_key: "8793c60b7e692be69412368db9db57dd"
server_key_iv: "999a207b830e3276eebaec8f7b4dd3bd"
server_key_version: "encryptionkey@0"
I20260806 16:11:28.725780 5141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "f19870b4111348e1aa16136d83ab01b2"
format_stamp: "Formatted at 2026-08-06 16:11:28 on dist-test-slave-7z32"
server_key: "8793c60b7e692be69412368db9db57dd"
server_key_iv: "999a207b830e3276eebaec8f7b4dd3bd"
server_key_version: "encryptionkey@0"
I20260806 16:11:28.732282 5141 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:11:28.736688 5165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:28.738917 5141 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260806 16:11:28.739176 5141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "f19870b4111348e1aa16136d83ab01b2"
format_stamp: "Formatted at 2026-08-06 16:11:28 on dist-test-slave-7z32"
server_key: "8793c60b7e692be69412368db9db57dd"
server_key_iv: "999a207b830e3276eebaec8f7b4dd3bd"
server_key_version: "encryptionkey@0"
I20260806 16:11:28.739353 5141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:28.763624 5141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:28.768239 5141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:28.768584 5141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:28.779150 5141 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34895
I20260806 16:11:28.779186 5232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34895 every 8 connection(s)
I20260806 16:11:28.780465 5141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:28.785151 5233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:28.786347 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 5141
I20260806 16:11:28.786612 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:28.787087 8388 external_mini_cluster.cc:1428] Setting key adb9ec21544301ccbe381ca793f17df7
I20260806 16:11:28.795410 5233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2: Bootstrap starting.
I20260806 16:11:28.800846 5233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:28.802064 5237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:28.789734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:41805 (local address 127.0.0.1:34895)
0806 16:11:28.790936 (+ 1202us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:28.790962 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:28.791032 (+ 70us) server_negotiation.cc:407] Connection header received
0806 16:11:28.792150 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:28.792183 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:28.792696 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:28.793154 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:28.793705 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:28.794807 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:28.796334 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:28.797007 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:28.797218 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:28.799408 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:28.799415 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:28.799439 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:28.800020 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:28.800106 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:28.800136 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:28.800329 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:28.800583 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 16:11:28.800777 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":223,"threads_started":1}
I20260806 16:11:28.802071 5233 log.cc:824] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:28.806694 5233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2: No bootstrap required, opened a new log
I20260806 16:11:28.808743 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:28.814518 5233 raft_consensus.cc:348] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19870b4111348e1aa16136d83ab01b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } }
I20260806 16:11:28.815006 5233 raft_consensus.cc:374] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:28.815090 5233 raft_consensus.cc:729] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f19870b4111348e1aa16136d83ab01b2, State: Initialized, Role: FOLLOWER
I20260806 16:11:28.815707 5233 consensus_queue.cc:260] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19870b4111348e1aa16136d83ab01b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } }
I20260806 16:11:28.815904 5233 raft_consensus.cc:388] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:28.816022 5233 raft_consensus.cc:482] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:28.816175 5233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:28.817677 5233 raft_consensus.cc:504] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19870b4111348e1aa16136d83ab01b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } }
I20260806 16:11:28.817984 5237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:28.811692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:51373 (local address 127.0.0.1:34895)
0806 16:11:28.811889 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:28.811895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:28.812012 (+ 117us) server_negotiation.cc:407] Connection header received
0806 16:11:28.812160 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:28.812164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:28.812242 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:28.812393 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:28.812867 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:28.813849 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:28.815298 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:28.815772 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:28.815939 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:28.817264 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:28.817268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:28.817271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:28.817289 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:28.817324 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:28.817339 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:28.817461 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:28.817718 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 16:11:28.817770 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 16:11:28.818228 5233 leader_election.cc:302] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f19870b4111348e1aa16136d83ab01b2; no voters:
I20260806 16:11:28.818758 5233 leader_election.cc:288] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:28.819142 5248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:28.819592 5248 raft_consensus.cc:686] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [term 1 LEADER]: Becoming Leader. State: Replica: f19870b4111348e1aa16136d83ab01b2, State: Running, Role: LEADER
I20260806 16:11:28.820122 5248 consensus_queue.cc:237] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19870b4111348e1aa16136d83ab01b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } }
I20260806 16:11:28.820710 5233 sys_catalog.cc:564] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:28.822635 5250 sys_catalog.cc:455] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f19870b4111348e1aa16136d83ab01b2. Latest consensus state: current_term: 1 leader_uuid: "f19870b4111348e1aa16136d83ab01b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19870b4111348e1aa16136d83ab01b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } } }
I20260806 16:11:28.822858 5250 sys_catalog.cc:458] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:28.823278 5249 sys_catalog.cc:455] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f19870b4111348e1aa16136d83ab01b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f19870b4111348e1aa16136d83ab01b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } } }
I20260806 16:11:28.823488 5249 sys_catalog.cc:458] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:28.823814 5264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:28.827896 5264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:28.835011 5264 catalog_manager.cc:1347] Generated new cluster ID: 03ce8559d5e247bcaeac17869a64dad3
I20260806 16:11:28.835170 5264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:28.862290 5264 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:28.864061 5264 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:28.873986 5264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f19870b4111348e1aa16136d83ab01b2: Generated new TSK 0
I20260806 16:11:28.875025 5264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:28.881541 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 5141
W20260806 16:11:28.905627 5117 connection.cc:535] client connection to 127.0.0.1:34895 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34895): Connection reset by peer (error 104)
2026-08-06T16:11:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1862 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:28 dist-test-slave-7z32 krb5kdc[5274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:28 dist-test-slave-7z32 krb5kdc[5274](info): set up 2 sockets
Aug 06 16:11:28 dist-test-slave-7z32 krb5kdc[5274](info): commencing operation
krb5kdc: starting...
W20260806 16:11:30.221211 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.002s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:30 dist-test-slave-7z32 krb5kdc[5274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:30.395671 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37651
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:30.562199 5290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:30.562675 5290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:30.562783 5290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:30.567431 5290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:30.567618 5290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:30.567672 5290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:30.567713 5290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:30.567749 5290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:30.574200 5290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:30.575944 5290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:30.577428 5290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:30.589639 5296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:30.589862 5297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:30.589928 5299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:30.590044 5290 server_base.cc:1034] running on GCE node
I20260806 16:11:30.591312 5290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:30.593019 5290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:30.594383 5290 hybrid_clock.cc:648] HybridClock initialized: now 1786032690594336 us; error 74 us; skew 500 ppm
Aug 06 16:11:30 dist-test-slave-7z32 krb5kdc[5274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032690, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:30.598812 5290 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:30.600351 5290 webserver.cc:460] Webserver started at http://127.0.0.1:41535/ using document root <none> and password file <none>
I20260806 16:11:30.601190 5290 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:30.601315 5290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:30.601637 5290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:30.605450 5290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "139278fb47124d5cb033fbef7730c3ba"
format_stamp: "Formatted at 2026-08-06 16:11:30 on dist-test-slave-7z32"
server_key: "b6266b01cfcdfd5002dce4c1ab61a83e"
server_key_iv: "45915b7fee12e2bf461a0110e5dcdc09"
server_key_version: "encryptionkey@0"
I20260806 16:11:30.606560 5290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "139278fb47124d5cb033fbef7730c3ba"
format_stamp: "Formatted at 2026-08-06 16:11:30 on dist-test-slave-7z32"
server_key: "b6266b01cfcdfd5002dce4c1ab61a83e"
server_key_iv: "45915b7fee12e2bf461a0110e5dcdc09"
server_key_version: "encryptionkey@0"
I20260806 16:11:30.612746 5290 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260806 16:11:30.617100 5307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:30.619796 5290 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 16:11:30.620074 5290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "139278fb47124d5cb033fbef7730c3ba"
format_stamp: "Formatted at 2026-08-06 16:11:30 on dist-test-slave-7z32"
server_key: "b6266b01cfcdfd5002dce4c1ab61a83e"
server_key_iv: "45915b7fee12e2bf461a0110e5dcdc09"
server_key_version: "encryptionkey@0"
I20260806 16:11:30.620326 5290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:30.627746 5290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:30.633059 5290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:30.633380 5290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:30.644780 5290 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37651
I20260806 16:11:30.644805 5381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37651 every 8 connection(s)
I20260806 16:11:30.646246 5290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:30.648007 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 5290
I20260806 16:11:30.648181 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:30.648518 8388 external_mini_cluster.cc:1428] Setting key 9c0c412be5e7d77a28f6ceeb814b8214
I20260806 16:11:30.650720 5382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:30.656864 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:30.651016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:35779 (local address 127.0.0.1:37651)
0806 16:11:30.651705 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:30.651738 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:30.651799 (+ 61us) server_negotiation.cc:407] Connection header received
0806 16:11:30.652749 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:30.652775 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:30.653220 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:30.653635 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:30.655014 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:30.655020 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:30.655036 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:30.655051 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:30.655131 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:30.655156 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:30.655344 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:30.655483 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 16:11:30.655761 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":205,"threads_started":1}
I20260806 16:11:30.659273 5382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba: Bootstrap starting.
I20260806 16:11:30.661902 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:30.662248 5382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:30.663203 5382 log.cc:824] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:30.666535 5382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba: No bootstrap required, opened a new log
I20260806 16:11:30.668057 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:30.665467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:38143 (local address 127.0.0.1:37651)
0806 16:11:30.665643 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:30.665649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:30.665899 (+ 250us) server_negotiation.cc:407] Connection header received
0806 16:11:30.666173 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:30.666177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:30.666225 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:30.666342 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:30.667553 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:30.667556 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:30.667558 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:30.667572 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:30.667603 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:30.667614 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:30.667743 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:30.667813 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 16:11:30.667879 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 16:11:30.671087 5382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139278fb47124d5cb033fbef7730c3ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } }
I20260806 16:11:30.671382 5382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:30.671448 5382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139278fb47124d5cb033fbef7730c3ba, State: Initialized, Role: FOLLOWER
I20260806 16:11:30.671940 5382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139278fb47124d5cb033fbef7730c3ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } }
I20260806 16:11:30.672075 5382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:30.672178 5382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:30.672303 5382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:30.673427 5382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139278fb47124d5cb033fbef7730c3ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } }
I20260806 16:11:30.673830 5382 leader_election.cc:302] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 139278fb47124d5cb033fbef7730c3ba; no voters:
I20260806 16:11:30.674221 5382 leader_election.cc:288] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:30.674546 5394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:30.675096 5394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [term 1 LEADER]: Becoming Leader. State: Replica: 139278fb47124d5cb033fbef7730c3ba, State: Running, Role: LEADER
I20260806 16:11:30.675670 5394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139278fb47124d5cb033fbef7730c3ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } }
I20260806 16:11:30.676091 5382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:30.677996 5396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 139278fb47124d5cb033fbef7730c3ba. Latest consensus state: current_term: 1 leader_uuid: "139278fb47124d5cb033fbef7730c3ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139278fb47124d5cb033fbef7730c3ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } } }
I20260806 16:11:30.678205 5396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:30.678618 5395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "139278fb47124d5cb033fbef7730c3ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139278fb47124d5cb033fbef7730c3ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37651 } } }
I20260806 16:11:30.679237 5395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba [sys.catalog]: This master's current role is: LEADER
W20260806 16:11:30.681826 5417 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:11:30.681929 5417 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:30.682071 5416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:30.682957 5416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:30.690274 5416 catalog_manager.cc:1347] Generated new cluster ID: 130475b7b9054fda83a16a2ed03e58b7
I20260806 16:11:30.690398 5416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:30.704248 5416 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:30.704851 5416 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:30.715305 5416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 139278fb47124d5cb033fbef7730c3ba: Generated new TSK 0
I20260806 16:11:30.716727 5416 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:30.728600 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 5290
2026-08-06T16:11:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1842 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:30 dist-test-slave-7z32 krb5kdc[5423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:30 dist-test-slave-7z32 krb5kdc[5423](info): set up 2 sockets
Aug 06 16:11:30 dist-test-slave-7z32 krb5kdc[5423](info): commencing operation
krb5kdc: starting...
W20260806 16:11:32.071678 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.001s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:32 dist-test-slave-7z32 krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:32.248713 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:32.401415 5438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:32.401926 5438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:32.402021 5438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:32.408874 5438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:32.409052 5438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:32.409102 5438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:32.409142 5438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:32.409204 5438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:32.420276 5438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:32.421939 5438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:32.423486 5438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:32.435005 5446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:32.435030 5450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:32.435030 5445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:32.435590 5438 server_base.cc:1034] running on GCE node
I20260806 16:11:32.436333 5438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:32.437836 5438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:32.441144 5438 hybrid_clock.cc:648] HybridClock initialized: now 1786032692441099 us; error 71 us; skew 500 ppm
Aug 06 16:11:32 dist-test-slave-7z32 krb5kdc[5423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032692, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:32.444665 5438 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:32.445814 5438 webserver.cc:460] Webserver started at http://127.0.0.1:41323/ using document root <none> and password file <none>
I20260806 16:11:32.446468 5438 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:32.446537 5438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:32.446731 5438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:32.449124 5438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "8574916170364f3cb659ab957dec81e5"
format_stamp: "Formatted at 2026-08-06 16:11:32 on dist-test-slave-7z32"
server_key: "d9dcc92f01a7b8b30cb772b4b4d7b267"
server_key_iv: "ac8f2148402fadeba3a533c36ddaa74f"
server_key_version: "encryptionkey@0"
I20260806 16:11:32.449697 5438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "8574916170364f3cb659ab957dec81e5"
format_stamp: "Formatted at 2026-08-06 16:11:32 on dist-test-slave-7z32"
server_key: "d9dcc92f01a7b8b30cb772b4b4d7b267"
server_key_iv: "ac8f2148402fadeba3a533c36ddaa74f"
server_key_version: "encryptionkey@0"
I20260806 16:11:32.454376 5438 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 16:11:32.457830 5458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:32.459915 5438 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:11:32.460120 5438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "8574916170364f3cb659ab957dec81e5"
format_stamp: "Formatted at 2026-08-06 16:11:32 on dist-test-slave-7z32"
server_key: "d9dcc92f01a7b8b30cb772b4b4d7b267"
server_key_iv: "ac8f2148402fadeba3a533c36ddaa74f"
server_key_version: "encryptionkey@0"
I20260806 16:11:32.460285 5438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:32.485865 5438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:32.489948 5438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:32.490199 5438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:32.499935 5438 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44477
I20260806 16:11:32.500095 5539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44477 every 8 connection(s)
I20260806 16:11:32.501238 5438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:32.505908 5540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:32.510442 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 5438
I20260806 16:11:32.510710 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:32.511085 8388 external_mini_cluster.cc:1428] Setting key f3f6e3052b8d9299269d589e9efd984d
I20260806 16:11:32.514472 5540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5: Bootstrap starting.
I20260806 16:11:32.518114 5540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:32.519434 5540 log.cc:824] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:32.523444 5540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5: No bootstrap required, opened a new log
Aug 06 16:11:32 dist-test-slave-7z32 krb5kdc[5423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:11:32.527859 5540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8574916170364f3cb659ab957dec81e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } }
I20260806 16:11:32.528239 5540 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:32.528337 5540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8574916170364f3cb659ab957dec81e5, State: Initialized, Role: FOLLOWER
I20260806 16:11:32.528961 5540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8574916170364f3cb659ab957dec81e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } }
I20260806 16:11:32.529160 5540 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:32.529268 5540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:32.529421 5540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:32.530923 5540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8574916170364f3cb659ab957dec81e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } }
I20260806 16:11:32.531404 5540 leader_election.cc:302] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8574916170364f3cb659ab957dec81e5; no voters:
I20260806 16:11:32.531787 5543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:32.513165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:47095 (local address 127.0.0.1:44477)
0806 16:11:32.513707 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:32.513734 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:32.513777 (+ 43us) server_negotiation.cc:407] Connection header received
0806 16:11:32.514702 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:32.514726 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:32.515254 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:32.515601 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:32.516903 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:32.518098 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:32.519676 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:32.520467 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:32.520723 (+ 256us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:32.524835 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:32.524843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:32.524866 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:32.524907 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:32.527674 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:32.528358 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:32.528364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:32.528371 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:32.528564 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:32.529062 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:32.529066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:32.529068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:32.529746 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:32.530009 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:32.530371 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 16:11:32.530654 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":198,"threads_started":1}
I20260806 16:11:32.531946 5540 leader_election.cc:288] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:32.532228 5545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:32.532554 5545 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [term 1 LEADER]: Becoming Leader. State: Replica: 8574916170364f3cb659ab957dec81e5, State: Running, Role: LEADER
I20260806 16:11:32.533016 5545 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8574916170364f3cb659ab957dec81e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } }
I20260806 16:11:32.533912 5540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:32.534785 5545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8574916170364f3cb659ab957dec81e5. Latest consensus state: current_term: 1 leader_uuid: "8574916170364f3cb659ab957dec81e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8574916170364f3cb659ab957dec81e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } } }
I20260806 16:11:32.535072 5545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:32.535862 5555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:32.536208 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:32.536224 5546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8574916170364f3cb659ab957dec81e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8574916170364f3cb659ab957dec81e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44477 } } }
I20260806 16:11:32.536399 5546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:32.540972 5555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:32.549105 5555 catalog_manager.cc:1347] Generated new cluster ID: 19bb966c92d14088afddcc97c4456ca8
I20260806 16:11:32.549213 5555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:32.554984 5543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:32.540457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:44779 (local address 127.0.0.1:44477)
0806 16:11:32.540616 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:32.540621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:32.540967 (+ 346us) server_negotiation.cc:407] Connection header received
0806 16:11:32.541178 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:32.541182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:32.541244 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:32.541305 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:32.542551 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:32.543581 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:32.545109 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:32.545529 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:32.545652 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:32.548222 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:32.548228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:32.548231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:32.548280 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:32.552678 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:32.553395 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:32.553404 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:32.553406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:32.553496 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:32.553872 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:32.553875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:32.553877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:32.554104 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:32.554313 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:32.554590 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 16:11:32.554710 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:11:32.571249 5555 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:32.572583 5555 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:32.585474 5555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8574916170364f3cb659ab957dec81e5: Generated new TSK 0
I20260806 16:11:32.586721 5555 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:32.602310 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 5438
W20260806 16:11:32.621342 5421 connection.cc:535] client connection to 127.0.0.1:44477 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44477): Connection reset by peer (error 104)
2026-08-06T16:11:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1873 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:32 dist-test-slave-7z32 krb5kdc[5581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:32 dist-test-slave-7z32 krb5kdc[5581](info): set up 2 sockets
Aug 06 16:11:32 dist-test-slave-7z32 krb5kdc[5581](info): commencing operation
krb5kdc: starting...
W20260806 16:11:33.919317 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.002s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:34 dist-test-slave-7z32 krb5kdc[5581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:34.091153 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:34.247364 5597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:34.247814 5597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:34.247870 5597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:34.252686 5597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:34.252791 5597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:34.252813 5597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:34.252833 5597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:34.252851 5597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:34.259215 5597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33055
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:34.260730 5597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:34.262126 5597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:11:34.273393 5597 server_base.cc:1034] running on GCE node
W20260806 16:11:34.273308 5605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:34.273339 5604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:34.273285 5607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:34.274502 5597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:34.276206 5597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:34.277463 5597 hybrid_clock.cc:648] HybridClock initialized: now 1786032694277433 us; error 53 us; skew 500 ppm
Aug 06 16:11:34 dist-test-slave-7z32 krb5kdc[5581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:34.291662 5597 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:34.293318 5597 webserver.cc:460] Webserver started at http://127.0.0.1:42561/ using document root <none> and password file <none>
I20260806 16:11:34.294095 5597 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:34.294174 5597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:34.294405 5597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:34.297202 5597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "0a5a58a2d49f412bab6502c732d0a589"
format_stamp: "Formatted at 2026-08-06 16:11:34 on dist-test-slave-7z32"
server_key: "f84c7047313c03d76ce28f65acb3b19a"
server_key_iv: "aaccc421c908e2cb6289c90b23d1f6ba"
server_key_version: "encryptionkey@0"
I20260806 16:11:34.297894 5597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "0a5a58a2d49f412bab6502c732d0a589"
format_stamp: "Formatted at 2026-08-06 16:11:34 on dist-test-slave-7z32"
server_key: "f84c7047313c03d76ce28f65acb3b19a"
server_key_iv: "aaccc421c908e2cb6289c90b23d1f6ba"
server_key_version: "encryptionkey@0"
I20260806 16:11:34.303371 5597 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 16:11:34.307135 5616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:34.309211 5597 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:34.309401 5597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "0a5a58a2d49f412bab6502c732d0a589"
format_stamp: "Formatted at 2026-08-06 16:11:34 on dist-test-slave-7z32"
server_key: "f84c7047313c03d76ce28f65acb3b19a"
server_key_iv: "aaccc421c908e2cb6289c90b23d1f6ba"
server_key_version: "encryptionkey@0"
I20260806 16:11:34.309548 5597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:34.334312 5597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:34.339013 5597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:34.339244 5597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:34.349334 5597 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33055
I20260806 16:11:34.349395 5688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33055 every 8 connection(s)
I20260806 16:11:34.350721 5597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:34.354087 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 5597
I20260806 16:11:34.354254 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:34.354543 5689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:34.354616 8388 external_mini_cluster.cc:1428] Setting key d2665a6d1b1629fd46c8a54f86999bb0
I20260806 16:11:34.363761 5689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589: Bootstrap starting.
I20260806 16:11:34.367568 5693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:34.356815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:42245 (local address 127.0.0.1:33055)
0806 16:11:34.357455 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:34.357477 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:34.357525 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:11:34.358536 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:34.358562 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:34.359157 (+ 595us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:34.359570 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:34.360149 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:34.361306 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:34.362789 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:34.363464 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:34.363615 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:34.364923 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:34.364934 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:34.364960 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:34.365738 (+ 778us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:34.365859 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:34.365900 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:34.366042 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:34.366301 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 16:11:34.366479 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":215,"threads_started":1}
I20260806 16:11:34.367624 5689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:34.368844 5689 log.cc:824] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:34.371623 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:34.371800 5689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589: No bootstrap required, opened a new log
I20260806 16:11:34.377084 5689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5a58a2d49f412bab6502c732d0a589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } }
I20260806 16:11:34.377463 5689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:34.377591 5689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a5a58a2d49f412bab6502c732d0a589, State: Initialized, Role: FOLLOWER
I20260806 16:11:34.378435 5689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5a58a2d49f412bab6502c732d0a589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } }
I20260806 16:11:34.378724 5689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:34.378860 5689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:34.378989 5689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:34.380061 5693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:34.374518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:43791 (local address 127.0.0.1:33055)
0806 16:11:34.374703 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:34.374708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:34.374986 (+ 278us) server_negotiation.cc:407] Connection header received
0806 16:11:34.375161 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:34.375163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:34.375225 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:34.375308 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:34.375740 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:34.376335 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:34.377520 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:34.377926 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:34.378030 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:34.379488 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:34.379492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:34.379495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:34.379514 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:34.379553 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:34.379565 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:34.379670 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:34.379840 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 16:11:34.379885 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 16:11:34.380224 5689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5a58a2d49f412bab6502c732d0a589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } }
I20260806 16:11:34.380694 5689 leader_election.cc:302] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a5a58a2d49f412bab6502c732d0a589; no voters:
I20260806 16:11:34.381102 5689 leader_election.cc:288] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:34.381284 5701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:34.381574 5701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [term 1 LEADER]: Becoming Leader. State: Replica: 0a5a58a2d49f412bab6502c732d0a589, State: Running, Role: LEADER
I20260806 16:11:34.382064 5701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5a58a2d49f412bab6502c732d0a589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } }
I20260806 16:11:34.382666 5689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:34.383760 5703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a5a58a2d49f412bab6502c732d0a589. Latest consensus state: current_term: 1 leader_uuid: "0a5a58a2d49f412bab6502c732d0a589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5a58a2d49f412bab6502c732d0a589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } } }
I20260806 16:11:34.383975 5703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:34.384310 5704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a5a58a2d49f412bab6502c732d0a589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5a58a2d49f412bab6502c732d0a589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33055 } } }
I20260806 16:11:34.384433 5704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:34.384531 5711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:34.388624 5711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:34.396821 5711 catalog_manager.cc:1347] Generated new cluster ID: 6248d52604644f9299273151c420350e
I20260806 16:11:34.396960 5711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:34.423250 5711 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:34.424710 5711 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:34.433220 5711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a5a58a2d49f412bab6502c732d0a589: Generated new TSK 0
I20260806 16:11:34.434202 5711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:34.439270 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 5597
W20260806 16:11:34.458500 5578 connection.cc:535] client connection to 127.0.0.1:33055 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33055): Connection reset by peer (error 104)
2026-08-06T16:11:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1836 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:34 dist-test-slave-7z32 krb5kdc[5729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:34 dist-test-slave-7z32 krb5kdc[5729](info): set up 2 sockets
Aug 06 16:11:34 dist-test-slave-7z32 krb5kdc[5729](info): commencing operation
krb5kdc: starting...
W20260806 16:11:35.817891 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.001s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:35 dist-test-slave-7z32 krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:11:35.997092 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38331
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:36.164850 5746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:36.165324 5746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:36.165421 5746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:36.170086 5746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:36.170243 5746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:36.170284 5746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:36.170325 5746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:36.170363 5746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:36.177397 5746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38331
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:36.179147 5746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:36.180959 5746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:36.194104 5753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:36.194252 5752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:36.194101 5755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:36.194630 5746 server_base.cc:1034] running on GCE node
I20260806 16:11:36.196400 5746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:36.198185 5746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:36.201648 5746 hybrid_clock.cc:648] HybridClock initialized: now 1786032696201573 us; error 86 us; skew 500 ppm
Aug 06 16:11:36 dist-test-slave-7z32 krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032696, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:36.206077 5746 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:11:36.207722 5746 webserver.cc:460] Webserver started at http://127.0.0.1:38807/ using document root <none> and password file <none>
I20260806 16:11:36.208541 5746 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:36.208626 5746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:36.208942 5746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:36.212325 5746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "17cc40392bc54b44ad3d2602fc395889"
format_stamp: "Formatted at 2026-08-06 16:11:36 on dist-test-slave-7z32"
server_key: "9c6c5d56269320a8f3e1e099bfc9665e"
server_key_iv: "d207f9193c91f8a6bd096025b95c6740"
server_key_version: "encryptionkey@0"
I20260806 16:11:36.213066 5746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "17cc40392bc54b44ad3d2602fc395889"
format_stamp: "Formatted at 2026-08-06 16:11:36 on dist-test-slave-7z32"
server_key: "9c6c5d56269320a8f3e1e099bfc9665e"
server_key_iv: "d207f9193c91f8a6bd096025b95c6740"
server_key_version: "encryptionkey@0"
I20260806 16:11:36.218950 5746 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:11:36.223068 5764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:36.225402 5746 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:11:36.225610 5746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "17cc40392bc54b44ad3d2602fc395889"
format_stamp: "Formatted at 2026-08-06 16:11:36 on dist-test-slave-7z32"
server_key: "9c6c5d56269320a8f3e1e099bfc9665e"
server_key_iv: "d207f9193c91f8a6bd096025b95c6740"
server_key_version: "encryptionkey@0"
I20260806 16:11:36.225770 5746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:36.232802 5746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:36.237704 5746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:36.237955 5746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:36.248917 5746 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38331
I20260806 16:11:36.249078 5832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38331 every 8 connection(s)
I20260806 16:11:36.250389 5746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:36.254427 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:36.260155 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 5746
I20260806 16:11:36.260327 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:36.260649 8388 external_mini_cluster.cc:1428] Setting key b646777c0cb90a82d9cbcab395e34c74
I20260806 16:11:36.261765 5836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889: Bootstrap starting.
I20260806 16:11:36.264689 5836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:36.265621 5836 log.cc:824] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:36.269480 5836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889: No bootstrap required, opened a new log
I20260806 16:11:36.269927 5839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:36.262623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:34383 (local address 127.0.0.1:38331)
0806 16:11:36.263401 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:36.263418 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:36.263463 (+ 45us) server_negotiation.cc:407] Connection header received
0806 16:11:36.264737 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:36.264830 (+ 93us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:36.265352 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:36.266308 (+ 956us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:36.267622 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:36.267630 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:36.267653 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:36.267668 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:36.267761 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:36.267794 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:36.267968 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:36.268329 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 16:11:36.268570 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":231,"threads_started":1}
I20260806 16:11:36.275542 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:36.276053 5836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17cc40392bc54b44ad3d2602fc395889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38331 } }
I20260806 16:11:36.276454 5836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:36.276559 5836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cc40392bc54b44ad3d2602fc395889, State: Initialized, Role: FOLLOWER
I20260806 16:11:36.277261 5836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17cc40392bc54b44ad3d2602fc395889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38331 } }
I20260806 16:11:36.277501 5836 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:36.277614 5836 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:36.277760 5836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:36.279724 5836 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17cc40392bc54b44ad3d2602fc395889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38331 } }
I20260806 16:11:36.280310 5836 leader_election.cc:302] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17cc40392bc54b44ad3d2602fc395889; no voters:
I20260806 16:11:36.280843 5836 leader_election.cc:288] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:36.280972 5848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:36.281005 5839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:36.278708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.194:43399 (local address 127.0.0.1:38331)
0806 16:11:36.278903 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:36.278909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:36.279235 (+ 326us) server_negotiation.cc:407] Connection header received
0806 16:11:36.279430 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:36.279433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:36.279479 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:36.279585 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:36.280533 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:36.280537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:36.280539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:36.280555 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:36.280591 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:36.280605 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:36.280720 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:36.280791 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 16:11:36.280846 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 16:11:36.281293 5848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [term 1 LEADER]: Becoming Leader. State: Replica: 17cc40392bc54b44ad3d2602fc395889, State: Running, Role: LEADER
I20260806 16:11:36.281740 5848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17cc40392bc54b44ad3d2602fc395889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38331 } }
I20260806 16:11:36.282918 5836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:36.283751 5848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17cc40392bc54b44ad3d2602fc395889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17cc40392bc54b44ad3d2602fc395889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38331 } } }
I20260806 16:11:36.283931 5848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:36.284200 5848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17cc40392bc54b44ad3d2602fc395889. Latest consensus state: current_term: 1 leader_uuid: "17cc40392bc54b44ad3d2602fc395889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17cc40392bc54b44ad3d2602fc395889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38331 } } }
I20260806 16:11:36.284451 5858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:36.284438 5848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:36.289618 5858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:36.300246 5858 catalog_manager.cc:1347] Generated new cluster ID: 3c88a4cdb30d45e4a06ca361e147a701
I20260806 16:11:36.300392 5858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:36.318856 5858 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:36.319453 5858 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:36.331831 5858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17cc40392bc54b44ad3d2602fc395889: Generated new TSK 0
I20260806 16:11:36.333063 5858 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:36.343147 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 5746
2026-08-06T16:11:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1899 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:36 dist-test-slave-7z32 krb5kdc[5879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:36 dist-test-slave-7z32 krb5kdc[5879](info): set up 2 sockets
Aug 06 16:11:36 dist-test-slave-7z32 krb5kdc[5879](info): commencing operation
krb5kdc: starting...
W20260806 16:11:37.710806 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.002s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:37 dist-test-slave-7z32 krb5kdc[5879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:11:37.887750 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:35781
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:35781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:38.028276 5895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:38.028759 5895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:38.028857 5895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:38.033545 5895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:38.033707 5895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:38.033749 5895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:38.033788 5895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:38.033826 5895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:38.040503 5895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:35781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:35781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:38.041893 5895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:38.043133 5895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:38.054679 5905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:38.054708 5902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:38.054682 5903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:38.055174 5895 server_base.cc:1034] running on GCE node
I20260806 16:11:38.056074 5895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:38.057693 5895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:38.058933 5895 hybrid_clock.cc:648] HybridClock initialized: now 1786032698058893 us; error 60 us; skew 500 ppm
Aug 06 16:11:38 dist-test-slave-7z32 krb5kdc[5879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032698, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:38.062960 5895 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:11:38.064338 5895 webserver.cc:460] Webserver started at http://127.8.49.62:38291/ using document root <none> and password file <none>
I20260806 16:11:38.065121 5895 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:38.065230 5895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:38.065527 5895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:38.068463 5895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "ce03deeba4ea40819cb05291f0f2d084"
format_stamp: "Formatted at 2026-08-06 16:11:38 on dist-test-slave-7z32"
server_key: "0ed282c94da7f46e6ee8cdf8c64a7cc5"
server_key_iv: "10e03a2002a27023e7581d9f54191383"
server_key_version: "encryptionkey@0"
I20260806 16:11:38.069227 5895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "ce03deeba4ea40819cb05291f0f2d084"
format_stamp: "Formatted at 2026-08-06 16:11:38 on dist-test-slave-7z32"
server_key: "0ed282c94da7f46e6ee8cdf8c64a7cc5"
server_key_iv: "10e03a2002a27023e7581d9f54191383"
server_key_version: "encryptionkey@0"
I20260806 16:11:38.074733 5895 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:11:38.078411 5915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:38.080751 5895 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:38.081022 5895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "ce03deeba4ea40819cb05291f0f2d084"
format_stamp: "Formatted at 2026-08-06 16:11:38 on dist-test-slave-7z32"
server_key: "0ed282c94da7f46e6ee8cdf8c64a7cc5"
server_key_iv: "10e03a2002a27023e7581d9f54191383"
server_key_version: "encryptionkey@0"
I20260806 16:11:38.081211 5895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:38.109472 5895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:38.114382 5895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:38.114681 5895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:38.127323 5895 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:35781
I20260806 16:11:38.127357 5976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:35781 every 8 connection(s)
I20260806 16:11:38.128827 5895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:38.130069 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 5895
I20260806 16:11:38.130379 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:38.130888 8388 external_mini_cluster.cc:1428] Setting key 24f8a8e3678dde4444c2e7d2ec6056ef
I20260806 16:11:38.134037 5977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:38.140833 5977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084: Bootstrap starting.
Aug 06 16:11:38 dist-test-slave-7z32 krb5kdc[5879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:38.143996 5977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:38.145188 5977 log.cc:824] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:38.148594 5977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084: No bootstrap required, opened a new log
I20260806 16:11:38.153630 5977 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce03deeba4ea40819cb05291f0f2d084" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 35781 } }
I20260806 16:11:38.153999 5977 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:38.154112 5977 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce03deeba4ea40819cb05291f0f2d084, State: Initialized, Role: FOLLOWER
I20260806 16:11:38.154323 5982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:38.133229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58147 (local address 127.8.49.62:35781)
0806 16:11:38.133957 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:38.133980 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:38.134038 (+ 58us) server_negotiation.cc:407] Connection header received
0806 16:11:38.135186 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:38.135217 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:38.135754 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:38.136239 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:38.137396 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:38.138721 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:38.139864 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:38.140503 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:38.140726 (+ 223us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:38.144826 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:38.144835 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:38.144861 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:38.144912 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:38.147827 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:38.148928 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:38.148937 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:38.148946 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:38.149139 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:38.149869 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:38.149873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:38.149875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:38.150587 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:38.150981 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:38.151953 (+ 972us) server_negotiation.cc:299] Negotiation successful
0806 16:11:38.152784 (+ 831us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":239,"threads_started":1}
I20260806 16:11:38.154783 5977 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce03deeba4ea40819cb05291f0f2d084" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 35781 } }
I20260806 16:11:38.155052 5977 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:38.155131 5977 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:38.155241 5977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:38.156591 5977 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce03deeba4ea40819cb05291f0f2d084" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 35781 } }
I20260806 16:11:38.157112 5977 leader_election.cc:302] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce03deeba4ea40819cb05291f0f2d084; no voters:
I20260806 16:11:38.157581 5977 leader_election.cc:288] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:38.157723 5984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:38.158010 5984 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [term 1 LEADER]: Becoming Leader. State: Replica: ce03deeba4ea40819cb05291f0f2d084, State: Running, Role: LEADER
I20260806 16:11:38.158638 5984 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce03deeba4ea40819cb05291f0f2d084" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 35781 } }
I20260806 16:11:38.159215 5977 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:38.159963 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:38.161157 5986 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce03deeba4ea40819cb05291f0f2d084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce03deeba4ea40819cb05291f0f2d084" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 35781 } } }
I20260806 16:11:38.161407 5986 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:38.162037 5985 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce03deeba4ea40819cb05291f0f2d084. Latest consensus state: current_term: 1 leader_uuid: "ce03deeba4ea40819cb05291f0f2d084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce03deeba4ea40819cb05291f0f2d084" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 35781 } } }
I20260806 16:11:38.162209 5985 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:38.162683 5998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:38.167624 5998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:38.177498 5982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:38.164783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51337 (local address 127.8.49.62:35781)
0806 16:11:38.165150 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:38.165155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:38.165829 (+ 674us) server_negotiation.cc:407] Connection header received
0806 16:11:38.165984 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:38.165987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:38.166069 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:38.166220 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:38.167343 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:38.168171 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:38.169588 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:38.170041 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:38.170221 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:38.172493 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:38.172498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:38.172501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:38.172548 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:38.175135 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:38.175875 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:38.175880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:38.175882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:38.175990 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:38.176383 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:38.176387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:38.176389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:38.176639 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:38.176827 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:38.177136 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 16:11:38.177269 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271}
I20260806 16:11:38.178215 5998 catalog_manager.cc:1347] Generated new cluster ID: 3aff0b9b0ba34dab83e6bc8e51b246a3
I20260806 16:11:38.178295 5998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:38.203881 5998 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:38.205247 5998 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:38.219144 5998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce03deeba4ea40819cb05291f0f2d084: Generated new TSK 0
I20260806 16:11:38.220376 5998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:38.227463 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 5895
W20260806 16:11:38.246016 5877 connection.cc:535] client connection to 127.8.49.62:35781 recv error: Network error: failed to read from TLS socket (remote: 127.8.49.62:35781): Connection reset by peer (error 104)
2026-08-06T16:11:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1888 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:38 dist-test-slave-7z32 krb5kdc[6016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:38 dist-test-slave-7z32 krb5kdc[6016](info): set up 2 sockets
Aug 06 16:11:38 dist-test-slave-7z32 krb5kdc[6016](info): commencing operation
krb5kdc: starting...
W20260806 16:11:39.613282 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.351s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:39 dist-test-slave-7z32 krb5kdc[6016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:11:39.802150 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:39643
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:39643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:39.988075 6032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:39.988533 6032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:39.988590 6032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:39.994239 6032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:39.994382 6032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:39.994406 6032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:39.994426 6032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:39.994444 6032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:40.003955 6032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:39643
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:39643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:40.005841 6032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:40.007436 6032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:40.020164 6039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:40.020859 6040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:40.021235 6042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:40.021539 6032 server_base.cc:1034] running on GCE node
I20260806 16:11:40.022555 6032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:40.024516 6032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:40.027964 6032 hybrid_clock.cc:648] HybridClock initialized: now 1786032700027434 us; error 540 us; skew 500 ppm
Aug 06 16:11:40 dist-test-slave-7z32 krb5kdc[6016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:40.040624 6032 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:11:40.042632 6032 webserver.cc:460] Webserver started at http://127.8.49.62:40793/ using document root <none> and password file <none>
I20260806 16:11:40.043740 6032 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:40.043855 6032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:40.044094 6032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:40.047297 6032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "15e85dff88c64e2180189dbaf38e87fd"
format_stamp: "Formatted at 2026-08-06 16:11:40 on dist-test-slave-7z32"
server_key: "ffd78d776ab377d5add68ff1b2e4f0ad"
server_key_iv: "cc6fac2549f3a7ec095f55b7d476640c"
server_key_version: "encryptionkey@0"
I20260806 16:11:40.047998 6032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "15e85dff88c64e2180189dbaf38e87fd"
format_stamp: "Formatted at 2026-08-06 16:11:40 on dist-test-slave-7z32"
server_key: "ffd78d776ab377d5add68ff1b2e4f0ad"
server_key_iv: "cc6fac2549f3a7ec095f55b7d476640c"
server_key_version: "encryptionkey@0"
I20260806 16:11:40.053890 6032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 16:11:40.058264 6051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:40.060415 6032 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:40.060598 6032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "15e85dff88c64e2180189dbaf38e87fd"
format_stamp: "Formatted at 2026-08-06 16:11:40 on dist-test-slave-7z32"
server_key: "ffd78d776ab377d5add68ff1b2e4f0ad"
server_key_iv: "cc6fac2549f3a7ec095f55b7d476640c"
server_key_version: "encryptionkey@0"
I20260806 16:11:40.060796 6032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:40.085318 6032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:40.090631 6032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:40.090963 6032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:40.101806 6032 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:39643
I20260806 16:11:40.101830 6116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:39643 every 8 connection(s)
I20260806 16:11:40.103363 6032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:40.107076 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 6032
I20260806 16:11:40.107251 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:40.107460 6117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:40.107632 8388 external_mini_cluster.cc:1428] Setting key d5fda75d40995dff87fca5db98ceda87
I20260806 16:11:40.115875 6117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd: Bootstrap starting.
I20260806 16:11:40.118821 6117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:40.120005 6117 log.cc:824] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:40.121292 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:40.109797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50471 (local address 127.8.49.62:39643)
0806 16:11:40.110427 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:40.110452 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:40.110507 (+ 55us) server_negotiation.cc:407] Connection header received
0806 16:11:40.111384 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:40.111413 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:40.111927 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:40.112367 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:40.113162 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:40.114721 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:40.116278 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:40.116993 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:40.117203 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:40.118703 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:40.118712 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:40.118735 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:40.119379 (+ 644us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:40.119460 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:40.119493 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:40.119659 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:40.120072 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 16:11:40.120263 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":182,"threads_started":1}
I20260806 16:11:40.124439 6117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd: No bootstrap required, opened a new log
I20260806 16:11:40.126317 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:40.130432 6117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e85dff88c64e2180189dbaf38e87fd" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39643 } }
I20260806 16:11:40.130885 6117 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:40.130987 6117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15e85dff88c64e2180189dbaf38e87fd, State: Initialized, Role: FOLLOWER
I20260806 16:11:40.131654 6117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e85dff88c64e2180189dbaf38e87fd" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39643 } }
I20260806 16:11:40.131845 6117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:40.131949 6117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:40.132082 6117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:40.133677 6117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e85dff88c64e2180189dbaf38e87fd" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39643 } }
I20260806 16:11:40.134241 6117 leader_election.cc:302] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15e85dff88c64e2180189dbaf38e87fd; no voters:
I20260806 16:11:40.134594 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:40.129327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51751 (local address 127.8.49.62:39643)
0806 16:11:40.129471 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:40.129477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:40.129813 (+ 336us) server_negotiation.cc:407] Connection header received
0806 16:11:40.129998 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:40.130002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:40.130058 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:40.130153 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:40.130464 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:40.131222 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:40.132174 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:40.132470 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:40.132587 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:40.133742 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:40.133746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:40.133749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:40.133773 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:40.133801 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:40.133812 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:40.133929 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:40.134280 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 16:11:40.134403 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:11:40.134784 6117 leader_election.cc:288] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:40.134989 6129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:40.135319 6129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [term 1 LEADER]: Becoming Leader. State: Replica: 15e85dff88c64e2180189dbaf38e87fd, State: Running, Role: LEADER
I20260806 16:11:40.135833 6129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e85dff88c64e2180189dbaf38e87fd" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39643 } }
I20260806 16:11:40.136626 6117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:40.137413 6129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15e85dff88c64e2180189dbaf38e87fd. Latest consensus state: current_term: 1 leader_uuid: "15e85dff88c64e2180189dbaf38e87fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e85dff88c64e2180189dbaf38e87fd" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39643 } } }
I20260806 16:11:40.137593 6129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:40.138114 6138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:40.138599 6130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15e85dff88c64e2180189dbaf38e87fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e85dff88c64e2180189dbaf38e87fd" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 39643 } } }
I20260806 16:11:40.138715 6130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:40.142851 6138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:40.152251 6138 catalog_manager.cc:1347] Generated new cluster ID: daa2b16fa4b842a7a1c208bda1a9c069
I20260806 16:11:40.152412 6138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:40.177855 6138 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:40.179378 6138 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:40.192677 6138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15e85dff88c64e2180189dbaf38e87fd: Generated new TSK 0
I20260806 16:11:40.193632 6138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:40.206380 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 6032
W20260806 16:11:40.224306 6014 connection.cc:535] client connection to 127.8.49.62:39643 recv error: Network error: failed to read from TLS socket (remote: 127.8.49.62:39643): Connection reset by peer (error 104)
2026-08-06T16:11:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1978 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:40 dist-test-slave-7z32 krb5kdc[6153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:40 dist-test-slave-7z32 krb5kdc[6153](info): set up 2 sockets
Aug 06 16:11:40 dist-test-slave-7z32 krb5kdc[6153](info): commencing operation
krb5kdc: starting...
W20260806 16:11:41.624883 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.384s user 0.004s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:41 dist-test-slave-7z32 krb5kdc[6153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:11:41.813185 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:43897
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:37071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:43897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:41.959304 6170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:41.959728 6170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:41.959781 6170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:41.964500 6170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:41.964623 6170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:41.964648 6170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:41.964668 6170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:41.964686 6170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:41.971396 6170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:37071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:43897
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:43897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:41.973094 6170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:41.974535 6170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:41.987267 6178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:41.987258 6180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:41.987267 6177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:41.987795 6170 server_base.cc:1034] running on GCE node
I20260806 16:11:41.988907 6170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:41.990921 6170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:41.992260 6170 hybrid_clock.cc:648] HybridClock initialized: now 1786032701992197 us; error 74 us; skew 500 ppm
Aug 06 16:11:41 dist-test-slave-7z32 krb5kdc[6153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032701, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:41.996505 6170 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:11:41.998075 6170 webserver.cc:460] Webserver started at http://127.8.49.62:40749/ using document root <none> and password file <none>
I20260806 16:11:41.999006 6170 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:41.999099 6170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:41.999397 6170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:42.002331 6170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "20ee0b44d5764496b3452fd65e97f7c4"
format_stamp: "Formatted at 2026-08-06 16:11:41 on dist-test-slave-7z32"
server_key: "49dcc63abb69b4c32b023958ac30e48b"
server_key_iv: "d3ca9e62b66c929c343dea48a96500fe"
server_key_version: "encryptionkey@0"
I20260806 16:11:42.003193 6170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "20ee0b44d5764496b3452fd65e97f7c4"
format_stamp: "Formatted at 2026-08-06 16:11:41 on dist-test-slave-7z32"
server_key: "49dcc63abb69b4c32b023958ac30e48b"
server_key_iv: "d3ca9e62b66c929c343dea48a96500fe"
server_key_version: "encryptionkey@0"
I20260806 16:11:42.009119 6170 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 16:11:42.013497 6192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:42.015915 6170 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 16:11:42.016184 6170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "20ee0b44d5764496b3452fd65e97f7c4"
format_stamp: "Formatted at 2026-08-06 16:11:41 on dist-test-slave-7z32"
server_key: "49dcc63abb69b4c32b023958ac30e48b"
server_key_iv: "d3ca9e62b66c929c343dea48a96500fe"
server_key_version: "encryptionkey@0"
I20260806 16:11:42.016382 6170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:42.024675 6170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:42.030416 6170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:42.030751 6170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:42.041503 6170 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:43897
I20260806 16:11:42.041599 6256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:43897 every 8 connection(s)
I20260806 16:11:42.042958 6170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:42.045701 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 6170
I20260806 16:11:42.045918 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:42.046337 8388 external_mini_cluster.cc:1428] Setting key 63f6ec1091439ee901281372861acea1
I20260806 16:11:42.048214 6257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:42.055312 6263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:42.048738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42861 (local address 127.8.49.62:43897)
0806 16:11:42.049388 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:42.049416 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:42.049475 (+ 59us) server_negotiation.cc:407] Connection header received
0806 16:11:42.050793 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:42.050863 (+ 70us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:42.051353 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:42.051916 (+ 563us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:42.053121 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:42.053129 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:42.053152 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:42.053167 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:42.053260 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:42.053295 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:42.053511 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:42.053726 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 16:11:42.053950 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":198,"threads_started":1}
I20260806 16:11:42.058275 6257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4: Bootstrap starting.
I20260806 16:11:42.060160 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:42.062676 6257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:42.064087 6257 log.cc:824] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:42.066685 6263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:42.063699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59967 (local address 127.8.49.62:43897)
0806 16:11:42.063923 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:42.063928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:42.064073 (+ 145us) server_negotiation.cc:407] Connection header received
0806 16:11:42.064226 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:42.064231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:42.064307 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:42.064456 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:42.065664 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:42.065668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:42.065670 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:42.065690 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:42.065727 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:42.065739 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:42.066136 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:42.066203 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 16:11:42.066263 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 16:11:42.068269 6257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4: No bootstrap required, opened a new log
I20260806 16:11:42.072940 6257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ee0b44d5764496b3452fd65e97f7c4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43897 } }
I20260806 16:11:42.073344 6257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:42.073449 6257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ee0b44d5764496b3452fd65e97f7c4, State: Initialized, Role: FOLLOWER
I20260806 16:11:42.074026 6257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ee0b44d5764496b3452fd65e97f7c4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43897 } }
I20260806 16:11:42.074231 6257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:42.074339 6257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:42.074482 6257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:42.075877 6257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ee0b44d5764496b3452fd65e97f7c4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43897 } }
I20260806 16:11:42.076285 6257 leader_election.cc:302] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20ee0b44d5764496b3452fd65e97f7c4; no voters:
I20260806 16:11:42.076666 6257 leader_election.cc:288] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:42.076946 6273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:42.077462 6273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [term 1 LEADER]: Becoming Leader. State: Replica: 20ee0b44d5764496b3452fd65e97f7c4, State: Running, Role: LEADER
I20260806 16:11:42.078001 6273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ee0b44d5764496b3452fd65e97f7c4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43897 } }
I20260806 16:11:42.078405 6257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:42.081168 6275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20ee0b44d5764496b3452fd65e97f7c4. Latest consensus state: current_term: 1 leader_uuid: "20ee0b44d5764496b3452fd65e97f7c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ee0b44d5764496b3452fd65e97f7c4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43897 } } }
I20260806 16:11:42.081261 6274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20ee0b44d5764496b3452fd65e97f7c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ee0b44d5764496b3452fd65e97f7c4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 43897 } } }
I20260806 16:11:42.081425 6275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:42.081444 6274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4 [sys.catalog]: This master's current role is: LEADER
W20260806 16:11:42.086478 6292 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:11:42.086649 6292 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:42.086937 6291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:42.088048 6291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:42.096524 6291 catalog_manager.cc:1347] Generated new cluster ID: 2f3e249007544cc2bfdff93f8dadc535
I20260806 16:11:42.096714 6291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:42.129530 6291 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:42.130141 6291 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:42.137643 6291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20ee0b44d5764496b3452fd65e97f7c4: Generated new TSK 0
I20260806 16:11:42.138914 6291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:42.144164 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 6170
2026-08-06T16:11:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1937 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:42 dist-test-slave-7z32 krb5kdc[6299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:42 dist-test-slave-7z32 krb5kdc[6299](info): set up 2 sockets
Aug 06 16:11:42 dist-test-slave-7z32 krb5kdc[6299](info): commencing operation
krb5kdc: starting...
W20260806 16:11:43.494176 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.001s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:43 dist-test-slave-7z32 krb5kdc[6299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:11:43.688706 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:41655
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:38403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:41655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:43.871963 6316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:43.872491 6316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:43.872833 6316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:43.878990 6316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:43.879133 6316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:43.879169 6316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:43.879191 6316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:43.879210 6316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:43.887080 6316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:38403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:41655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:41655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:43.889338 6316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:43.891333 6316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:43.905582 6323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:43.905702 6325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:43.905651 6316 server_base.cc:1034] running on GCE node
W20260806 16:11:43.905576 6322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:43.906903 6316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:43.908627 6316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:43.909919 6316 hybrid_clock.cc:648] HybridClock initialized: now 1786032703909878 us; error 79 us; skew 500 ppm
Aug 06 16:11:43 dist-test-slave-7z32 krb5kdc[6299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032703, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:43.915572 6316 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:11:43.917443 6316 webserver.cc:460] Webserver started at http://127.8.49.62:43189/ using document root <none> and password file <none>
I20260806 16:11:43.918490 6316 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:43.918594 6316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:43.919008 6316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:43.922739 6316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "6a17553cf59a4b8dbbe4346256b14417"
format_stamp: "Formatted at 2026-08-06 16:11:43 on dist-test-slave-7z32"
server_key: "61a8c87bdb7a42a80dfb1176099672ed"
server_key_iv: "1f43c9bf2852ac9ea60367cbead02d04"
server_key_version: "encryptionkey@0"
I20260806 16:11:43.923931 6316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "6a17553cf59a4b8dbbe4346256b14417"
format_stamp: "Formatted at 2026-08-06 16:11:43 on dist-test-slave-7z32"
server_key: "61a8c87bdb7a42a80dfb1176099672ed"
server_key_iv: "1f43c9bf2852ac9ea60367cbead02d04"
server_key_version: "encryptionkey@0"
I20260806 16:11:43.931253 6316 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 16:11:43.935791 6333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:43.938068 6316 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:11:43.938410 6316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "6a17553cf59a4b8dbbe4346256b14417"
format_stamp: "Formatted at 2026-08-06 16:11:43 on dist-test-slave-7z32"
server_key: "61a8c87bdb7a42a80dfb1176099672ed"
server_key_iv: "1f43c9bf2852ac9ea60367cbead02d04"
server_key_version: "encryptionkey@0"
I20260806 16:11:43.938604 6316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:43.963538 6316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:43.970279 6316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:43.970541 6316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:43.982730 6316 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:41655
I20260806 16:11:43.982781 6406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:41655 every 8 connection(s)
I20260806 16:11:43.984310 6316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:43.985960 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 6316
I20260806 16:11:43.986145 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:43.986497 8388 external_mini_cluster.cc:1428] Setting key 4b82e251f150688227d13b5c23bc58c7
I20260806 16:11:43.989544 6408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:43.999600 6408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417: Bootstrap starting.
Aug 06 16:11:43 dist-test-slave-7z32 krb5kdc[6299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:44.003304 6408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:44.004302 6408 log.cc:824] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:44.007294 6408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417: No bootstrap required, opened a new log
I20260806 16:11:44.010812 6412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:43.988706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33173 (local address 127.8.49.62:41655)
0806 16:11:43.989569 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:43.989594 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:43.989683 (+ 89us) server_negotiation.cc:407] Connection header received
0806 16:11:43.990782 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:43.990811 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:43.991467 (+ 656us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:43.991921 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:43.993296 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:43.994815 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:43.996757 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:43.997707 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:43.997909 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:44.002525 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:44.002536 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:44.002564 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:44.002628 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:44.006064 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:44.006889 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:44.006900 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:44.006912 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:44.007093 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:44.007443 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:44.007448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:44.007450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:44.008087 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:44.008372 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:44.008741 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 16:11:44.009205 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":314,"threads_started":1}
I20260806 16:11:44.011840 6408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a17553cf59a4b8dbbe4346256b14417" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41655 } }
I20260806 16:11:44.012199 6408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:44.012253 6408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a17553cf59a4b8dbbe4346256b14417, State: Initialized, Role: FOLLOWER
I20260806 16:11:44.012756 6408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a17553cf59a4b8dbbe4346256b14417" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41655 } }
I20260806 16:11:44.012919 6408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:44.012961 6408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:44.013095 6408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:44.014568 6408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a17553cf59a4b8dbbe4346256b14417" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41655 } }
I20260806 16:11:44.015054 6408 leader_election.cc:302] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a17553cf59a4b8dbbe4346256b14417; no voters:
I20260806 16:11:44.015622 6408 leader_election.cc:288] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:44.015725 6414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:44.015935 6414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [term 1 LEADER]: Becoming Leader. State: Replica: 6a17553cf59a4b8dbbe4346256b14417, State: Running, Role: LEADER
I20260806 16:11:44.016286 6414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a17553cf59a4b8dbbe4346256b14417" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41655 } }
I20260806 16:11:44.016470 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:44.017150 6408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:44.018567 6416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a17553cf59a4b8dbbe4346256b14417. Latest consensus state: current_term: 1 leader_uuid: "6a17553cf59a4b8dbbe4346256b14417" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a17553cf59a4b8dbbe4346256b14417" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41655 } } }
I20260806 16:11:44.018784 6416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:44.019133 6415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a17553cf59a4b8dbbe4346256b14417" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a17553cf59a4b8dbbe4346256b14417" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41655 } } }
I20260806 16:11:44.019248 6415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:44.019311 6425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:44.024837 6425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:44.032564 6412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:44.020033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38989 (local address 127.8.49.62:41655)
0806 16:11:44.020216 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:44.020223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:44.020691 (+ 468us) server_negotiation.cc:407] Connection header received
0806 16:11:44.020876 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:44.020883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:44.020954 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:44.021100 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:44.022016 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:44.023753 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:44.025209 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:44.025598 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:44.025752 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:44.027990 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:44.027994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:44.027996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:44.028027 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:44.029985 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:44.030756 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:44.030762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:44.030765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:44.030980 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:44.031431 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:44.031437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:44.031439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:44.031696 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:44.031879 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:44.032177 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 16:11:44.032343 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 16:11:44.033209 6425 catalog_manager.cc:1347] Generated new cluster ID: 008d006772154780912fa863608e0b1c
I20260806 16:11:44.033288 6425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:44.042891 6425 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:44.044366 6425 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:44.054404 6425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a17553cf59a4b8dbbe4346256b14417: Generated new TSK 0
I20260806 16:11:44.055768 6425 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:44.063800 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 6316
2026-08-06T16:11:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1941 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:44 dist-test-slave-7z32 krb5kdc[6449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:44 dist-test-slave-7z32 krb5kdc[6449](info): set up 2 sockets
Aug 06 16:11:44 dist-test-slave-7z32 krb5kdc[6449](info): commencing operation
krb5kdc: starting...
W20260806 16:11:45.463541 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.008s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:45 dist-test-slave-7z32 krb5kdc[6449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:11:45.693279 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:40783
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:40783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:45.903816 6465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:45.904241 6465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:45.904297 6465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:45.910620 6465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:45.910741 6465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:45.910766 6465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:45.910786 6465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:45.910809 6465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:45.918474 6465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:40783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:40783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:45.920022 6465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:45.921365 6465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:45.934774 6470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:45.934777 6471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:45.934774 6473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:45.935762 6465 server_base.cc:1034] running on GCE node
I20260806 16:11:45.936578 6465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:45.938305 6465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:45.939726 6465 hybrid_clock.cc:648] HybridClock initialized: now 1786032705939679 us; error 78 us; skew 500 ppm
Aug 06 16:11:45 dist-test-slave-7z32 krb5kdc[6449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:45.943831 6465 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:11:45.945178 6465 webserver.cc:460] Webserver started at http://127.8.49.62:35781/ using document root <none> and password file <none>
I20260806 16:11:45.945885 6465 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:45.945998 6465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:45.946281 6465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:45.949443 6465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "7ef96a2b730f4105856b2c1afd67e849"
format_stamp: "Formatted at 2026-08-06 16:11:45 on dist-test-slave-7z32"
server_key: "e7feb7fd5b7909d0b102094444e344d5"
server_key_iv: "9d0a87795726410353fac0081a29e3cd"
server_key_version: "encryptionkey@0"
I20260806 16:11:45.950222 6465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "7ef96a2b730f4105856b2c1afd67e849"
format_stamp: "Formatted at 2026-08-06 16:11:45 on dist-test-slave-7z32"
server_key: "e7feb7fd5b7909d0b102094444e344d5"
server_key_iv: "9d0a87795726410353fac0081a29e3cd"
server_key_version: "encryptionkey@0"
I20260806 16:11:45.955952 6465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 16:11:45.959481 6480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:45.961123 6465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:11:45.961275 6465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "7ef96a2b730f4105856b2c1afd67e849"
format_stamp: "Formatted at 2026-08-06 16:11:45 on dist-test-slave-7z32"
server_key: "e7feb7fd5b7909d0b102094444e344d5"
server_key_iv: "9d0a87795726410353fac0081a29e3cd"
server_key_version: "encryptionkey@0"
I20260806 16:11:45.961457 6465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:45.985178 6465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:45.990350 6465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:45.990758 6465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:46.003903 6465 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:40783
I20260806 16:11:46.003939 6561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:40783 every 8 connection(s)
I20260806 16:11:46.005512 6465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:46.009733 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 6465
I20260806 16:11:46.009917 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:46.010114 6562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:46.010566 8388 external_mini_cluster.cc:1428] Setting key cdd49dd7715323fa9b28236e6ec96eff
I20260806 16:11:46.020526 6562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849: Bootstrap starting.
I20260806 16:11:46.023765 6562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:46.024869 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:46.012908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52109 (local address 127.8.49.62:40783)
0806 16:11:46.013623 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:46.013643 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:46.013695 (+ 52us) server_negotiation.cc:407] Connection header received
0806 16:11:46.015626 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:46.015676 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:46.016469 (+ 793us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:46.017055 (+ 586us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:46.017651 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:46.018974 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:46.020139 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:46.020894 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:46.021107 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:46.022304 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:46.022312 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:46.022344 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:46.022994 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:46.023082 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:46.023114 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:46.023246 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:46.023433 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 16:11:46.023746 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":193,"threads_started":1}
I20260806 16:11:46.024911 6562 log.cc:824] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:46.027695 6562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849: No bootstrap required, opened a new log
I20260806 16:11:46.029353 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:46.032560 6562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef96a2b730f4105856b2c1afd67e849" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40783 } }
I20260806 16:11:46.032895 6562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:46.032969 6562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ef96a2b730f4105856b2c1afd67e849, State: Initialized, Role: FOLLOWER
I20260806 16:11:46.033891 6562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef96a2b730f4105856b2c1afd67e849" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40783 } }
I20260806 16:11:46.034129 6562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:46.034245 6562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:46.034476 6562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:46.035889 6562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef96a2b730f4105856b2c1afd67e849" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40783 } }
I20260806 16:11:46.036378 6562 leader_election.cc:302] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ef96a2b730f4105856b2c1afd67e849; no voters:
I20260806 16:11:46.036873 6562 leader_election.cc:288] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:46.036959 6574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:46.037228 6574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [term 1 LEADER]: Becoming Leader. State: Replica: 7ef96a2b730f4105856b2c1afd67e849, State: Running, Role: LEADER
I20260806 16:11:46.037629 6574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef96a2b730f4105856b2c1afd67e849" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40783 } }
I20260806 16:11:46.038609 6562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:46.039175 6576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ef96a2b730f4105856b2c1afd67e849. Latest consensus state: current_term: 1 leader_uuid: "7ef96a2b730f4105856b2c1afd67e849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef96a2b730f4105856b2c1afd67e849" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40783 } } }
I20260806 16:11:46.039377 6576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:46.039162 6577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ef96a2b730f4105856b2c1afd67e849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef96a2b730f4105856b2c1afd67e849" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40783 } } }
I20260806 16:11:46.039606 6577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:46.039631 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:46.032470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49259 (local address 127.8.49.62:40783)
0806 16:11:46.032669 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:46.032681 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:46.033003 (+ 322us) server_negotiation.cc:407] Connection header received
0806 16:11:46.033843 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:46.033925 (+ 82us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:46.034023 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:46.034158 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:46.034758 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:46.035456 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:46.036685 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:46.037047 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:46.037208 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:46.038294 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:46.038300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:46.038305 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:46.038342 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:46.038380 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:46.038397 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:46.038474 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:46.039046 (+ 572us) server_negotiation.cc:299] Negotiation successful
0806 16:11:46.039131 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:11:46.040197 6579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:46.044744 6579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:46.052649 6579 catalog_manager.cc:1347] Generated new cluster ID: 7cf9d1d547dc479da2eaba990fbc1c2a
I20260806 16:11:46.052776 6579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:46.072485 6579 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:46.074388 6579 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:46.084367 6579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ef96a2b730f4105856b2c1afd67e849: Generated new TSK 0
I20260806 16:11:46.085670 6579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:46.096431 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 6465
2026-08-06T16:11:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2011 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:46 dist-test-slave-7z32 krb5kdc[6602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:46 dist-test-slave-7z32 krb5kdc[6602](info): set up 2 sockets
Aug 06 16:11:46 dist-test-slave-7z32 krb5kdc[6602](info): commencing operation
krb5kdc: starting...
W20260806 16:11:47.451197 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.002s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:47 dist-test-slave-7z32 krb5kdc[6602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:11:47.629083 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:41603
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:41603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:47.796984 6620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:47.797430 6620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:47.797484 6620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:47.802161 6620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:47.802276 6620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:47.802299 6620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:47.802317 6620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:47.802335 6620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:47.809978 6620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:41603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:41603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:47.812081 6620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:47.813532 6620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:47.825258 6631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:47.825399 6628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:47.825258 6629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:47.825693 6620 server_base.cc:1034] running on GCE node
I20260806 16:11:47.826674 6620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:47.828498 6620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:47.829871 6620 hybrid_clock.cc:648] HybridClock initialized: now 1786032707829817 us; error 75 us; skew 500 ppm
Aug 06 16:11:47 dist-test-slave-7z32 krb5kdc[6602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032707, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:47.834218 6620 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:11:47.835922 6620 webserver.cc:460] Webserver started at http://127.8.49.62:38687/ using document root <none> and password file <none>
I20260806 16:11:47.836742 6620 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:47.836825 6620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:47.837106 6620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:47.840082 6620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "063302a0865e4fbea12c7d31db3282c6"
format_stamp: "Formatted at 2026-08-06 16:11:47 on dist-test-slave-7z32"
server_key: "925b30d7656a6fcd4c4313fe54b6072d"
server_key_iv: "5f914f6e1d1adbf08e562cc6fd68b210"
server_key_version: "encryptionkey@0"
I20260806 16:11:47.841010 6620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "063302a0865e4fbea12c7d31db3282c6"
format_stamp: "Formatted at 2026-08-06 16:11:47 on dist-test-slave-7z32"
server_key: "925b30d7656a6fcd4c4313fe54b6072d"
server_key_iv: "5f914f6e1d1adbf08e562cc6fd68b210"
server_key_version: "encryptionkey@0"
I20260806 16:11:47.847611 6620 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:11:47.852672 6640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:47.855017 6620 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:11:47.855289 6620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "063302a0865e4fbea12c7d31db3282c6"
format_stamp: "Formatted at 2026-08-06 16:11:47 on dist-test-slave-7z32"
server_key: "925b30d7656a6fcd4c4313fe54b6072d"
server_key_iv: "5f914f6e1d1adbf08e562cc6fd68b210"
server_key_version: "encryptionkey@0"
I20260806 16:11:47.855495 6620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:47.862273 6620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:47.867305 6620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:47.867599 6620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:47.878321 6620 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:41603
I20260806 16:11:47.878363 6716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:41603 every 8 connection(s)
I20260806 16:11:47.879797 6620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:47.882669 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 6620
I20260806 16:11:47.882900 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:47.883303 8388 external_mini_cluster.cc:1428] Setting key b8711afd4f4045e7666939d47e9c2d07
I20260806 16:11:47.883625 6717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:47.891718 6723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:47.885706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50775 (local address 127.8.49.62:41603)
0806 16:11:47.886366 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:47.886390 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:47.886449 (+ 59us) server_negotiation.cc:407] Connection header received
0806 16:11:47.887754 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:47.887787 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:47.888282 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:47.888831 (+ 549us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:47.889938 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:47.889944 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:47.889960 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:47.889969 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:47.890029 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:47.890052 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:47.890258 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:47.890340 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 16:11:47.890557 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":183,"threads_started":1}
I20260806 16:11:47.892570 6717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6: Bootstrap starting.
I20260806 16:11:47.895560 6717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:47.896262 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:47.896531 6717 log.cc:824] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:47.899806 6717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6: No bootstrap required, opened a new log
I20260806 16:11:47.902334 6723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:47.899635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43521 (local address 127.8.49.62:41603)
0806 16:11:47.899846 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:47.899853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:47.900224 (+ 371us) server_negotiation.cc:407] Connection header received
0806 16:11:47.900407 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:47.900410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:47.900479 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:47.900585 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:47.901630 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:47.901634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:47.901636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:11:47.901653 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:47.901695 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:11:47.901708 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:47.901817 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:47.901926 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 16:11:47.901972 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 16:11:47.904796 6717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063302a0865e4fbea12c7d31db3282c6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41603 } }
I20260806 16:11:47.905136 6717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:11:47.905242 6717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 063302a0865e4fbea12c7d31db3282c6, State: Initialized, Role: FOLLOWER
I20260806 16:11:47.905774 6717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063302a0865e4fbea12c7d31db3282c6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41603 } }
I20260806 16:11:47.905936 6717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:11:47.906041 6717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:11:47.906152 6717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:47.907347 6717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063302a0865e4fbea12c7d31db3282c6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41603 } }
I20260806 16:11:47.907743 6717 leader_election.cc:302] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 063302a0865e4fbea12c7d31db3282c6; no voters:
I20260806 16:11:47.908294 6717 leader_election.cc:288] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:11:47.908385 6736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:47.908695 6736 raft_consensus.cc:686] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [term 1 LEADER]: Becoming Leader. State: Replica: 063302a0865e4fbea12c7d31db3282c6, State: Running, Role: LEADER
I20260806 16:11:47.909216 6736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063302a0865e4fbea12c7d31db3282c6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41603 } }
I20260806 16:11:47.910075 6717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:47.911612 6738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 063302a0865e4fbea12c7d31db3282c6. Latest consensus state: current_term: 1 leader_uuid: "063302a0865e4fbea12c7d31db3282c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063302a0865e4fbea12c7d31db3282c6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41603 } } }
I20260806 16:11:47.911679 6739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "063302a0865e4fbea12c7d31db3282c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063302a0865e4fbea12c7d31db3282c6" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 41603 } } }
I20260806 16:11:47.911957 6738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:47.912113 6739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:47.913233 6755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:47.917680 6755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:47.925968 6755 catalog_manager.cc:1347] Generated new cluster ID: 9d02f992549e476da7dddbd9a7a0d5fd
I20260806 16:11:47.926090 6755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:47.938570 6755 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:47.939062 6755 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:47.951818 6755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 063302a0865e4fbea12c7d31db3282c6: Generated new TSK 0
I20260806 16:11:47.952723 6755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:47.962867 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 6620
2026-08-06T16:11:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1868 ms)
[----------] 18 tests from AuthTokenIssuingTest (34347 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:48 dist-test-slave-7z32 krb5kdc[6768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:48 dist-test-slave-7z32 krb5kdc[6768](info): set up 2 sockets
Aug 06 16:11:48 dist-test-slave-7z32 krb5kdc[6768](info): commencing operation
krb5kdc: starting...
W20260806 16:11:49.347395 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.012s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:49 dist-test-slave-7z32 krb5kdc[6768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:11:49.548684 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42695
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:42695,127.8.49.61:39779,127.8.49.60:46691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:49.716022 6789 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:11:49.716583 6789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:49.716642 6789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:49.716746 6789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:49.722455 6789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:49.722604 6789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:49.722626 6789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:49.722645 6789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:49.722663 6789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:49.732728 6789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:42695,127.8.49.61:39779,127.8.49.60:46691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:42695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:49.734782 6789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:49.736624 6789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:49.749176 6797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:49.749277 6789 server_base.cc:1034] running on GCE node
W20260806 16:11:49.749197 6803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:49.749207 6795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:49.750531 6789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:49.752313 6789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:49.753638 6789 hybrid_clock.cc:648] HybridClock initialized: now 1786032709753594 us; error 66 us; skew 500 ppm
Aug 06 16:11:49 dist-test-slave-7z32 krb5kdc[6768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:49.758162 6789 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:11:49.759943 6789 webserver.cc:460] Webserver started at http://127.8.49.62:33319/ using document root <none> and password file <none>
I20260806 16:11:49.760948 6789 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:49.761098 6789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:49.761339 6789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:49.764170 6789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "1fd6c4c688144ff5a4b37623a86b914d"
format_stamp: "Formatted at 2026-08-06 16:11:49 on dist-test-slave-7z32"
server_key: "9de381ae76c707cfc675587d539ffe9f"
server_key_iv: "15375662c57a28675f424924af84ebbe"
server_key_version: "encryptionkey@0"
I20260806 16:11:49.764932 6789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "1fd6c4c688144ff5a4b37623a86b914d"
format_stamp: "Formatted at 2026-08-06 16:11:49 on dist-test-slave-7z32"
server_key: "9de381ae76c707cfc675587d539ffe9f"
server_key_iv: "15375662c57a28675f424924af84ebbe"
server_key_version: "encryptionkey@0"
I20260806 16:11:49.770387 6789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 16:11:49.774246 6812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:49.776356 6789 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 16:11:49.776604 6789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "1fd6c4c688144ff5a4b37623a86b914d"
format_stamp: "Formatted at 2026-08-06 16:11:49 on dist-test-slave-7z32"
server_key: "9de381ae76c707cfc675587d539ffe9f"
server_key_iv: "15375662c57a28675f424924af84ebbe"
server_key_version: "encryptionkey@0"
I20260806 16:11:49.776774 6789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:49.794482 6789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:49.799998 6789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:49.800222 6789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:49.810392 6789 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:42695
I20260806 16:11:49.810940 6890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:42695 every 8 connection(s)
I20260806 16:11:49.811992 6789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:49.817044 6891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:49.821707 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 6789
I20260806 16:11:49.821929 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:49.822340 8388 external_mini_cluster.cc:1428] Setting key b7c9ab845ced2de5ec5f725779b5d4b5
I20260806 16:11:49.823385 6891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.49.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.61@KRBTEST.COM" created.
Aug 06 16:11:49 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:49.849902 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:49.827348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.8.49.62:42695)
0806 16:11:49.828187 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:49.828263 (+ 76us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:49.828334 (+ 71us) server_negotiation.cc:407] Connection header received
0806 16:11:49.829799 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:49.829810 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:49.830196 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:49.830519 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:49.831750 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:49.833415 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:49.834955 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:49.835751 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:49.835929 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:49.841271 (+ 5342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:49.841279 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:49.841298 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:49.841326 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:49.844298 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:49.845242 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:49.845254 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:49.845259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:49.845509 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:49.845930 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:49.845936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:49.845940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:49.846796 (+ 856us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:49.847020 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:49.847952 (+ 932us) server_negotiation.cc:299] Negotiation successful
0806 16:11:49.848389 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":648,"thread_start_us":232,"threads_started":1}
I20260806 16:11:49.849905 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:49.827347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42695 (local address 127.0.0.1:48628)
0806 16:11:49.828092 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:49.828159 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:49.829319 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:49.830572 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:49.830579 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:49.831156 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:49.831517 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:49.831535 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:49.833655 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:49.833659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:49.834707 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:49.834723 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:49.835991 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:49.835995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:49.836187 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:49.837320 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:49.837342 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:49.840987 (+ 3645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:49.844632 (+ 3645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:49.844646 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:49.844660 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:49.845025 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:49.845645 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:49.845647 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:49.845649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:49.845798 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:49.847024 (+ 1226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:49.847034 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:49.847331 (+ 297us) client_negotiation.cc:769] Sending connection context
0806 16:11:49.847759 (+ 428us) client_negotiation.cc:240] Negotiation successful
0806 16:11:49.848226 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":6,"thread_start_us":201,"threads_started":1}
I20260806 16:11:49.854959 6891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:11:49.856393 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:49.855824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:39779 (local address 127.0.0.1:42024)
0806 16:11:49.856162 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:49.856277 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260806 16:11:49.856683 6814 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
I20260806 16:11:49.858196 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:49.857803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:39779 (local address 127.0.0.1:42034)
0806 16:11:49.857987 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:49.858077 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 16:11:49.858492 6891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:39779: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab.
Entry for principal kudu/127.8.49.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab.
I20260806 16:11:49.868008 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.61:39779
--webserver_interface=127.8.49.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab
--principal=kudu/127.8.49.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:42695,127.8.49.61:39779,127.8.49.60:46691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:11:49.907940 6891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } attempt: 1
I20260806 16:11:49.909432 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:49.908916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:39779 (local address 127.0.0.1:42038)
0806 16:11:49.909182 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:49.909304 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 16:11:49.910506 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:49.910185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:39779 (local address 127.0.0.1:42042)
0806 16:11:49.910358 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:49.910431 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 16:11:49.910756 6891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:39779: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
I20260806 16:11:49.979097 6891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } attempt: 2
I20260806 16:11:49.980551 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:49.979886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:39779 (local address 127.0.0.1:42056)
0806 16:11:49.980217 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:49.980371 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 16:11:49.981997 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:49.981572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:39779 (local address 127.0.0.1:42062)
0806 16:11:49.981763 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:49.981871 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 16:11:49.982259 6891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:39779: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
W20260806 16:11:50.038913 6901 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:11:50.039445 6901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:50.039475 6901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:50.039561 6901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:50.044456 6901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:50.044584 6901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:50.044616 6901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:50.044637 6901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:50.044656 6901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:50.051501 6901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:42695,127.8.49.61:39779,127.8.49.60:46691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab
--principal=kudu/127.8.49.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.61:39779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.49.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:50.053292 6901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:50.055819 6901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:50.068269 6909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:50.068343 6913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:50.068343 6910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:50.069154 6901 server_base.cc:1034] running on GCE node
I20260806 16:11:50.070079 6901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:50.071852 6901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:50.073122 6901 hybrid_clock.cc:648] HybridClock initialized: now 1786032710073084 us; error 68 us; skew 500 ppm
I20260806 16:11:50.073539 6891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } attempt: 3
I20260806 16:11:50.074749 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.074153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:39779 (local address 127.0.0.1:42070)
0806 16:11:50.074513 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.074614 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
I20260806 16:11:50.075699 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.075463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:39779 (local address 127.0.0.1:42076)
0806 16:11:50.075578 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.075640 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 16:11:50.075930 6891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:39779: Network error: Client connection negotiation failed: client connection to 127.8.49.61:39779: connect: Connection refused (error 111)
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032710, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:50.077656 6901 init.cc:377] Logged in from keytab as kudu/127.8.49.61@KRBTEST.COM (short username kudu)
I20260806 16:11:50.079010 6901 webserver.cc:460] Webserver started at http://127.8.49.61:44243/ using document root <none> and password file <none>
I20260806 16:11:50.079695 6901 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:50.079797 6901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:50.080269 6901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:50.083197 6901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/data/instance:
uuid: "9070a46ad55541b98e0e5e9b5507587b"
format_stamp: "Formatted at 2026-08-06 16:11:50 on dist-test-slave-7z32"
server_key: "17672aa8abbf3f68209c810e830c9092"
server_key_iv: "b1ad8ae00487adcf59f3696eedaafd0a"
server_key_version: "encryptionkey@0"
I20260806 16:11:50.083922 6901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/wal/instance:
uuid: "9070a46ad55541b98e0e5e9b5507587b"
format_stamp: "Formatted at 2026-08-06 16:11:50 on dist-test-slave-7z32"
server_key: "17672aa8abbf3f68209c810e830c9092"
server_key_iv: "b1ad8ae00487adcf59f3696eedaafd0a"
server_key_version: "encryptionkey@0"
I20260806 16:11:50.089365 6901 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:11:50.093230 6922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:50.096087 6901 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 16:11:50.096319 6901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/wal
uuid: "9070a46ad55541b98e0e5e9b5507587b"
format_stamp: "Formatted at 2026-08-06 16:11:50 on dist-test-slave-7z32"
server_key: "17672aa8abbf3f68209c810e830c9092"
server_key_iv: "b1ad8ae00487adcf59f3696eedaafd0a"
server_key_version: "encryptionkey@0"
I20260806 16:11:50.096493 6901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:50.129722 6901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:50.135300 6901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:50.135550 6901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:50.147590 6901 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.61:39779
I20260806 16:11:50.147943 6994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.61:39779 every 8 connection(s)
I20260806 16:11:50.149135 6901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
I20260806 16:11:50.152171 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 6901
I20260806 16:11:50.152338 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-1/wal/instance
I20260806 16:11:50.152652 8388 external_mini_cluster.cc:1428] Setting key 3d4d0082819515420ab6ab24a926bab8
I20260806 16:11:50.153221 6996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:50.158257 6996 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.49.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.60@KRBTEST.COM" created.
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032710, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:50.183403 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.160823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48640 (local address 127.8.49.62:42695)
0806 16:11:50.161098 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.161107 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.162369 (+ 1262us) server_negotiation.cc:407] Connection header received
0806 16:11:50.164518 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.164522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.164599 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.164733 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.167368 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.168083 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.169760 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.170094 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.170238 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.174672 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.174676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.174679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.174716 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.177207 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.178508 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.178512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.178522 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.178655 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.179775 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.179778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.179780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.180297 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.180439 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.183055 (+ 2616us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.183241 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 16:11:50.184096 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.161169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42695 (local address 127.0.0.1:48640)
0806 16:11:50.162026 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.162166 (+ 140us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:50.164082 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:50.165220 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:50.165236 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:50.165926 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:50.166981 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.167005 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.168356 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.168361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.169551 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.169571 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.170426 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.170435 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.170872 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.171918 (+ 1046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:50.171946 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:50.174196 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:50.177481 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.177498 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.177537 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.178226 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.178957 (+ 731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.178965 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.178969 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.179314 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.180634 (+ 1320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:50.180649 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:50.181359 (+ 710us) client_negotiation.cc:769] Sending connection context
0806 16:11:50.181731 (+ 372us) client_negotiation.cc:240] Negotiation successful
0806 16:11:50.182342 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":171,"threads_started":1}
I20260806 16:11:50.186678 6996 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032710, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab.
Entry for principal kudu/127.8.49.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab.
I20260806 16:11:50.200868 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.60:46691
--webserver_interface=127.8.49.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab
--principal=kudu/127.8.49.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:42695,127.8.49.61:39779,127.8.49.60:46691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:11:50.203369 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.187818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42080 (local address 127.8.49.61:39779)
0806 16:11:50.188338 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.188356 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.188391 (+ 35us) server_negotiation.cc:407] Connection header received
0806 16:11:50.188461 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.188470 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.188704 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.188867 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.189551 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.191122 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.192626 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.193371 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.193528 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.197610 (+ 4082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.197641 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.197661 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.197689 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.200949 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.201621 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.201630 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.201631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.201733 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.202057 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.202060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.202062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.202502 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.202693 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.203021 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.203177 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":254,"threads_started":1}
I20260806 16:11:50.203356 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.187714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:39779 (local address 127.0.0.1:42080)
0806 16:11:50.187905 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.187925 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:50.188077 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:50.188937 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:50.188943 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:50.189219 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:50.189379 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.189387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.191331 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.191336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.192418 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.192432 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.193654 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.193658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.193768 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.195227 (+ 1459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:50.195250 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:50.197237 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:50.201223 (+ 3986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.201230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.201232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.201475 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.201840 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.201845 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.201848 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.201936 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.202622 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:50.202625 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:50.202723 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 16:11:50.202909 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 16:11:50.203107 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 16:11:50.206434 6996 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:11:50.207748 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.207255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38804)
0806 16:11:50.207514 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.207637 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260806 16:11:50.208073 6925 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
I20260806 16:11:50.209403 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.209129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38806)
0806 16:11:50.209263 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.209326 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 16:11:50.209599 6996 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:46691: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
I20260806 16:11:50.219133 6891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } attempt: 4
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
I20260806 16:11:50.231496 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.220176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:39779 (local address 127.0.0.1:42092)
0806 16:11:50.220489 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.220510 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:50.220715 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:50.221047 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:50.221050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:50.221306 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:50.221448 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.221454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.222334 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.222337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.223243 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.223257 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.223850 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.223854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.223989 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.224640 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:50.224653 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:50.226375 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:50.228814 (+ 2439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.228818 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.228820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.229091 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.230086 (+ 995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.230090 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.230091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.230202 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.230803 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:50.230806 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:50.231017 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 16:11:50.231141 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 16:11:50.231327 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260806 16:11:50.231631 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.220324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42092 (local address 127.8.49.61:39779)
0806 16:11:50.220488 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.220494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.220622 (+ 128us) server_negotiation.cc:407] Connection header received
0806 16:11:50.220864 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.220867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.220920 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.221016 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.221582 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.222192 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.223392 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.223699 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.223822 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.226589 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.226593 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.226595 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.226628 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.228624 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.229690 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.229698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.229700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.229920 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.230331 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.230335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.230337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.230652 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.230789 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.231239 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.231435 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 16:11:50.233268 6891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:11:50.234212 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.233912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38808)
0806 16:11:50.234064 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.234131 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 16:11:50.234957 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.234712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38814)
0806 16:11:50.234854 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.234908 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 16:11:50.235129 6891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:46691: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
I20260806 16:11:50.259035 6996 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } attempt: 1
I20260806 16:11:50.260548 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.259947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38826)
0806 16:11:50.260232 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.260394 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260806 16:11:50.261780 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.261443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38840)
0806 16:11:50.261580 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.261708 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 16:11:50.261945 6996 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:46691: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
I20260806 16:11:50.294546 6891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } attempt: 1
I20260806 16:11:50.296092 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.295554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38848)
0806 16:11:50.295833 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.295957 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 16:11:50.297219 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.296905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38856)
0806 16:11:50.297066 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.297144 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 16:11:50.297465 6891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:46691: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
I20260806 16:11:50.330179 6996 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } attempt: 2
I20260806 16:11:50.331794 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.331300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38860)
0806 16:11:50.331557 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.331666 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 16:11:50.333053 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.332717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38862)
0806 16:11:50.332910 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.332978 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260806 16:11:50.333390 6996 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:46691: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
W20260806 16:11:50.353992 7003 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:11:50.354336 7003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:50.354362 7003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:50.354451 7003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:50.358747 7003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:50.358909 7003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:50.358940 7003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:50.359031 7003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:50.359086 7003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:50.364749 6891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } attempt: 2
I20260806 16:11:50.365208 7003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:42695,127.8.49.61:39779,127.8.49.60:46691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab
--principal=kudu/127.8.49.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.60:46691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.49.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:50.365821 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.365466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38874)
0806 16:11:50.365654 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.365729 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 16:11:50.366701 7003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:50.367048 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.366727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38876)
0806 16:11:50.366916 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.366979 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 16:11:50.367274 6891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:46691: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
I20260806 16:11:50.368036 7003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:50.379159 7009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:50.379300 7010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:50.379300 7012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:50.379942 7003 server_base.cc:1034] running on GCE node
I20260806 16:11:50.380800 7003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:50.382355 7003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:50.383873 7003 hybrid_clock.cc:648] HybridClock initialized: now 1786032710383834 us; error 76 us; skew 500 ppm
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032710, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:50.388041 7003 init.cc:377] Logged in from keytab as kudu/127.8.49.60@KRBTEST.COM (short username kudu)
I20260806 16:11:50.389401 7003 webserver.cc:460] Webserver started at http://127.8.49.60:46573/ using document root <none> and password file <none>
I20260806 16:11:50.390151 7003 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:50.390228 7003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:50.390470 7003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:50.393342 7003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/data/instance:
uuid: "3997209206a04c878bfa0f605296e923"
format_stamp: "Formatted at 2026-08-06 16:11:50 on dist-test-slave-7z32"
server_key: "d39065d8bd9cb2e254797a815035cdef"
server_key_iv: "716aba2977dea961b842fd78824352f6"
server_key_version: "encryptionkey@0"
I20260806 16:11:50.394120 7003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/wal/instance:
uuid: "3997209206a04c878bfa0f605296e923"
format_stamp: "Formatted at 2026-08-06 16:11:50 on dist-test-slave-7z32"
server_key: "d39065d8bd9cb2e254797a815035cdef"
server_key_iv: "716aba2977dea961b842fd78824352f6"
server_key_version: "encryptionkey@0"
I20260806 16:11:50.399974 7003 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 16:11:50.405786 7020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:50.409252 7003 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260806 16:11:50.409556 7003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/wal
uuid: "3997209206a04c878bfa0f605296e923"
format_stamp: "Formatted at 2026-08-06 16:11:50 on dist-test-slave-7z32"
server_key: "d39065d8bd9cb2e254797a815035cdef"
server_key_iv: "716aba2977dea961b842fd78824352f6"
server_key_version: "encryptionkey@0"
I20260806 16:11:50.409807 7003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:50.424669 6996 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } attempt: 3
I20260806 16:11:50.425976 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.425552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38880)
0806 16:11:50.425786 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.425875 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 16:11:50.427282 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.426934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38892)
0806 16:11:50.427139 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.427207 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260806 16:11:50.427455 6996 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:46691: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
I20260806 16:11:50.452486 7003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:50.457995 7003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:50.458384 7003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:50.467540 6891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } attempt: 3
I20260806 16:11:50.468578 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.468210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38900)
0806 16:11:50.468389 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.468453 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:46691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 16:11:50.469475 7003 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.60:46691
I20260806 16:11:50.469760 7086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.60:46691 every 8 connection(s)
I20260806 16:11:50.471343 7003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
I20260806 16:11:50.475356 7088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:50.475983 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 7003
I20260806 16:11:50.476143 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786032364843570-8388-0/minicluster-data/master-2/wal/instance
I20260806 16:11:50.476528 8388 external_mini_cluster.cc:1428] Setting key f9ba4ff297b698c87e5350ab7a1fe7c5
I20260806 16:11:50.481274 7088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032710, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:50.494628 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.469269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38906)
0806 16:11:50.469518 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.469544 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:50.469776 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:50.473912 (+ 4136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:50.473916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:50.474123 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:50.474255 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.474261 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.476215 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.476222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.477081 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.477098 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.479005 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.479010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.479133 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.484920 (+ 5787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:50.484937 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:50.486946 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:50.490614 (+ 3668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.490618 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.490620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.490985 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.491557 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.491561 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.491564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.491719 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.493103 (+ 1384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:50.493110 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:50.493280 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 16:11:50.493500 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 16:11:50.494437 (+ 937us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 16:11:50.495958 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.477514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.8.49.60:46691)
0806 16:11:50.478378 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.478390 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.478410 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:11:50.478464 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.478468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.478538 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.478687 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.479800 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.480629 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.481894 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.482246 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.482362 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.487976 (+ 5614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.487984 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.487989 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.488042 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.490256 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.491028 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.491037 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.491047 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.491267 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.491662 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.491666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.491668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.492900 (+ 1232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.493179 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.493899 (+ 720us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.494340 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":723,"spinlock_wait_cycles":22016,"thread_start_us":462,"threads_started":1}
I20260806 16:11:50.495956 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.471178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38906 (local address 127.8.49.60:46691)
0806 16:11:50.471841 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.471861 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.471908 (+ 47us) server_negotiation.cc:407] Connection header received
0806 16:11:50.472962 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.472988 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.473461 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.473892 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.474366 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.475941 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.477308 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.478768 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.479037 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.487201 (+ 8164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.487212 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.487229 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.487272 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.490431 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.491113 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.491117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.491119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.491283 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.491857 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.491860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.491862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.492900 (+ 1038us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.493089 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.493901 (+ 812us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.494443 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":457,"spinlock_wait_cycles":4992,"thread_start_us":228,"threads_started":1}
I20260806 16:11:50.497816 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.483498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:42695 (local address 127.0.0.1:48648)
0806 16:11:50.483966 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.484032 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:50.484176 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:50.484705 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:50.484713 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:50.485225 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:50.485371 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.485377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.486244 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.486251 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.487078 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.487092 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.487734 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.487738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.487860 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.488486 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:50.488509 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:50.491741 (+ 3232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:50.494922 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.494937 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.494947 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.495305 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.495723 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.495726 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.495728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.495829 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.496671 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:50.496678 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:50.496835 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 16:11:50.497342 (+ 507us) client_negotiation.cc:240] Negotiation successful
0806 16:11:50.497596 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":193,"threads_started":1}
I20260806 16:11:50.498102 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.483712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48648 (local address 127.8.49.62:42695)
0806 16:11:50.483891 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.483895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.484154 (+ 259us) server_negotiation.cc:407] Connection header received
0806 16:11:50.484335 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.484339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.484461 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.484564 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.485592 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.486116 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.487269 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.487625 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.487739 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.491928 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.491933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.491936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.491965 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.494259 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.495465 (+ 1206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.495470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.495471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.495575 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.495974 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.495980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.495982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.496486 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.496592 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.497646 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.497932 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 16:11:50.499647 7088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:11:50.504782 6891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d: Bootstrap starting.
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032710, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:50.509521 6891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:50.509755 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.500212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:39779 (local address 127.0.0.1:42106)
0806 16:11:50.500567 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.500582 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:50.500677 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:50.500977 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:50.500979 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:50.501179 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:50.501288 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.501294 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.502271 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.502275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.503372 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.503383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.503929 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.503933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.504039 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.504614 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:50.504635 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:50.505984 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:50.508044 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.508048 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.508050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.508278 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.508589 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.508593 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.508603 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.508700 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.509127 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:50.509131 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:50.509250 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 16:11:50.509378 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 16:11:50.509566 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285}
I20260806 16:11:50.509755 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.500433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42106 (local address 127.8.49.61:39779)
0806 16:11:50.500560 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.500565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.500626 (+ 61us) server_negotiation.cc:407] Connection header received
0806 16:11:50.500799 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.500803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.500858 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.500950 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.501420 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.502148 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.503495 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.503792 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.503907 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.506154 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.506163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.506165 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.506192 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.507908 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.508397 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.508401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.508403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.508491 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.508798 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.508802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.508804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.509022 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.509135 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.509451 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.509579 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 16:11:50.511072 6891 log.cc:824] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:50.511102 7088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:11:50.514930 6891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d: No bootstrap required, opened a new log
I20260806 16:11:50.515631 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.501821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48664 (local address 127.8.49.62:42695)
0806 16:11:50.502192 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.502198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.502221 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:11:50.502276 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.502280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.502349 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.502478 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.503390 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.504296 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.505634 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.505983 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.506074 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.510183 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.510188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.510192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.510232 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.512319 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.513630 (+ 1311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.513635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.513637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.513968 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.514476 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.514480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.514482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.514773 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.515079 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.515265 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.515424 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032710, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
I20260806 16:11:50.520509 6891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } }
I20260806 16:11:50.520865 6891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd6c4c688144ff5a4b37623a86b914d, State: Initialized, Role: FOLLOWER
I20260806 16:11:50.521281 6891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } }
I20260806 16:11:50.522100 7095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } } }
I20260806 16:11:50.522526 7095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:50.523257 6891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:50.523656 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.511739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38916)
0806 16:11:50.512152 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.512165 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:50.512353 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:50.512759 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:50.512762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:50.512967 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:50.513113 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.513121 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.514523 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.514527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.515321 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.515334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.515925 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.515930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.516047 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.516721 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:50.516743 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:50.518500 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:50.521317 (+ 2817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.521323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.521326 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.521649 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.521976 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.521980 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.521982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.522073 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.522754 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:50.522757 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:50.522927 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 16:11:50.523110 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 16:11:50.523485 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":145}
I20260806 16:11:50.523656 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.511771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38916 (local address 127.8.49.60:46691)
0806 16:11:50.512055 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.512061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.512288 (+ 227us) server_negotiation.cc:407] Connection header received
0806 16:11:50.512530 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.512535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.512610 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.512697 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.513258 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.514370 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.515464 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.515761 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.515874 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.518636 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.518641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.518643 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.518666 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.521164 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.521778 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.521781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.521784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.521876 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.522192 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.522197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.522201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.522595 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.522714 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.523165 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.523417 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":100}
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
I20260806 16:11:50.528667 7088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923: Bootstrap starting.
I20260806 16:11:50.531793 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.517968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.8.49.61:39779)
0806 16:11:50.518155 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.518160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.518194 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:11:50.518254 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.518257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.518320 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.518444 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.519482 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.520400 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.521589 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.521896 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.521990 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.525903 (+ 3913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.525910 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.525915 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.525965 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.528790 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.529422 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.529426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.529428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.529549 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.529911 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.529915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.529916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.530241 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.530518 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.531444 (+ 926us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.531607 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 16:11:50.532819 7112 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 16:11:50.532905 7088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923: Neither blocks nor log segments found. Creating new log.
W20260806 16:11:50.533078 7112 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:50.533962 7088 log.cc:824] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:50.530700 6789 master_runner.cc:186] Error getting master registration for 127.8.49.61:39779: OK, instance_id { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" instance_seqno: 1786032710143973 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:11:50.535849 6789 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:11:50.536010 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:50.537392 6852 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1fd6c4c688144ff5a4b37623a86b914d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48664
I20260806 16:11:50.537508 7088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923: No bootstrap required, opened a new log
I20260806 16:11:50.537711 6852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 16:11:50.537808 6852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:50.539476 6852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } }
I20260806 16:11:50.540999 6852 leader_election.cc:288] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [CANDIDATE]: Term 1 election: Requested vote from peers 9070a46ad55541b98e0e5e9b5507587b (127.8.49.61:39779), 3997209206a04c878bfa0f605296e923 (127.8.49.60:46691)
I20260806 16:11:50.544036 7088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } }
I20260806 16:11:50.544817 7088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3997209206a04c878bfa0f605296e923, State: Initialized, Role: FOLLOWER
I20260806 16:11:50.545678 7088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } }
I20260806 16:11:50.548779 7088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:50.549222 7055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1fd6c4c688144ff5a4b37623a86b914d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3997209206a04c878bfa0f605296e923"
I20260806 16:11:50.550050 7055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [term 0 FOLLOWER]: Advancing to term 1
W20260806 16:11:50.552851 6820 tablet.cc:2369] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:11:50.553578 7055 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd6c4c688144ff5a4b37623a86b914d in term 1.
I20260806 16:11:50.554472 6814 leader_election.cc:302] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd6c4c688144ff5a4b37623a86b914d, 3997209206a04c878bfa0f605296e923; no voters:
I20260806 16:11:50.554997 7095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:50.555346 7095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [term 1 LEADER]: Becoming Leader. State: Replica: 1fd6c4c688144ff5a4b37623a86b914d, State: Running, Role: LEADER
I20260806 16:11:50.555833 7095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } }
I20260806 16:11:50.560415 7095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd6c4c688144ff5a4b37623a86b914d. Latest consensus state: current_term: 1 leader_uuid: "1fd6c4c688144ff5a4b37623a86b914d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } } }
I20260806 16:11:50.560689 7095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:50.561487 7140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:11:50.565716 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.544816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42120 (local address 127.8.49.61:39779)
0806 16:11:50.544984 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.544990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.547320 (+ 2330us) server_negotiation.cc:407] Connection header received
0806 16:11:50.547746 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.547750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.547828 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.547897 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.550115 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.550987 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.553162 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.553597 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.553697 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.557614 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.557619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.557622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.557663 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.559902 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.563423 (+ 3521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.563428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.563431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.563560 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.564129 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.564135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.564140 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.564396 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.564557 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.565236 (+ 679us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.565392 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:11:50.567384 7140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:50.570621 6996 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } attempt: 4
W20260806 16:11:50.574385 7137 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 16:11:50.576347 7055 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [term 1 FOLLOWER]: Refusing update from remote peer 1fd6c4c688144ff5a4b37623a86b914d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:50.577385 7138 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:50.566815 7003 master_runner.cc:186] Error getting master registration for 127.8.49.61:39779: OK, instance_id { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" instance_seqno: 1786032710143973 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:11:50.580850 7003 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:11:50.581763 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.546211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38924 (local address 127.8.49.60:46691)
0806 16:11:50.546385 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.546393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.566722 (+ 20329us) server_negotiation.cc:407] Connection header received
0806 16:11:50.566961 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.566964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.567032 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.567094 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.568158 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.568957 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.570198 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.570556 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.570652 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.572907 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.572912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.572917 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.572952 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.575498 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.576528 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.576534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.576564 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.576780 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.577566 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.577570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.577572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.578165 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.578518 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.581360 (+ 2842us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.581542 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:11:50.582751 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.547209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48668 (local address 127.8.49.62:42695)
0806 16:11:50.547361 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.547367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.547884 (+ 517us) server_negotiation.cc:407] Connection header received
0806 16:11:50.548250 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.548255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.548327 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.548393 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.562053 (+ 13660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.562888 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.564191 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.564571 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.564671 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.576109 (+ 11438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.576114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.576118 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.576163 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.578354 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.579292 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.579299 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.579301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.579408 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.579899 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.579903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.579904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.580173 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.580326 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.581493 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.581818 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 16:11:50.574596 7137 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:50.586123 7121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fd6c4c688144ff5a4b37623a86b914d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } } }
I20260806 16:11:50.586987 7121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:50.587620 7144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd6c4c688144ff5a4b37623a86b914d. Latest consensus state: current_term: 1 leader_uuid: "1fd6c4c688144ff5a4b37623a86b914d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } } }
I20260806 16:11:50.587769 7144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 16:11:50 dist-test-slave-7z32 krb5kdc[6768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032710, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
I20260806 16:11:50.594727 7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fd6c4c688144ff5a4b37623a86b914d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } } }
I20260806 16:11:50.594983 7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:50.594727 7142 mvcc.cc:205] Tried to move back new op lower bound from 7315589982506811392 to 7315589982441816064. Current Snapshot: MvccSnapshot[applied={T|T < 7315589982506811392}]
I20260806 16:11:50.596312 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.571556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:46691 (local address 127.0.0.1:38928)
0806 16:11:50.572045 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:50.572073 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:50.572200 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:50.573452 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:50.573456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:50.573754 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:50.573907 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.573919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.576076 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.576081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.576981 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:50.576997 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.578306 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.578310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:50.578446 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.586312 (+ 7866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:50.586339 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:50.588532 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:50.593780 (+ 5248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.593785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.593789 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.594145 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.594530 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:50.594534 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:50.594536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:50.594721 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:50.595480 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:50.595484 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:50.595715 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 16:11:50.595852 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 16:11:50.596080 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267}
I20260806 16:11:50.596258 7144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fd6c4c688144ff5a4b37623a86b914d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } } }
I20260806 16:11:50.596455 7144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:50.597040 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:50.572408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38928 (local address 127.8.49.60:46691)
0806 16:11:50.572575 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.572580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.572595 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:11:50.572642 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.572645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.572694 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.572778 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.574979 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.575895 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.577600 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:50.577993 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:50.578116 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:50.588844 (+ 10728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:50.588850 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:50.588853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:50.588900 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:50.593569 (+ 4669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.594292 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.594296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.594299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.594407 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:50.594914 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:50.594922 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:50.594926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:50.595304 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:50.595474 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:50.596673 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0806 16:11:50.596854 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 16:11:50.598547 7140 catalog_manager.cc:1347] Generated new cluster ID: 28deca68e7044400afc7efb4dcb7b69e
I20260806 16:11:50.598646 7140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:50.601827 6996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b: Bootstrap starting.
I20260806 16:11:50.603018 7145 mvcc.cc:205] Tried to move back new op lower bound from 7315589982506811392 to 7315589982441816064. Current Snapshot: MvccSnapshot[applied={T|T < 7315589982506811392}]
I20260806 16:11:50.608649 6996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:50.614522 6996 log.cc:824] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:50.621958 6996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b: No bootstrap required, opened a new log
I20260806 16:11:50.622597 7140 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:50.626104 7140 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:50.634878 6996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } }
I20260806 16:11:50.635277 6996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9070a46ad55541b98e0e5e9b5507587b, State: Initialized, Role: FOLLOWER
I20260806 16:11:50.635718 6996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } }
W20260806 16:11:50.637840 6929 tablet.cc:2369] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:11:50.637806 6959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1fd6c4c688144ff5a4b37623a86b914d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9070a46ad55541b98e0e5e9b5507587b"
I20260806 16:11:50.638127 6959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:50.638553 6996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:50.639802 6959 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd6c4c688144ff5a4b37623a86b914d in term 1.
I20260806 16:11:50.640771 6958 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [term 1 FOLLOWER]: Refusing update from remote peer 1fd6c4c688144ff5a4b37623a86b914d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:50.642042 7095 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:50.648821 7150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fd6c4c688144ff5a4b37623a86b914d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } } }
I20260806 16:11:50.649025 7150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:50.649130 7150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fd6c4c688144ff5a4b37623a86b914d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } } }
I20260806 16:11:50.649214 7150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:50.655326 7140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d: Generated new TSK 0
I20260806 16:11:50.656811 7140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:11:50.650226 7162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd6c4c688144ff5a4b37623a86b914d. Latest consensus state: current_term: 1 leader_uuid: "1fd6c4c688144ff5a4b37623a86b914d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } } }
I20260806 16:11:50.659488 7162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:50.661007 7165 mvcc.cc:205] Tried to move back new op lower bound from 7315589982688473088 to 7315589982441816064. Current Snapshot: MvccSnapshot[applied={T|T < 7315589982506811392}]
W20260806 16:11:50.664716 7171 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:11:50.667984 7171 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:50.668355 7138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fd6c4c688144ff5a4b37623a86b914d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd6c4c688144ff5a4b37623a86b914d" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 42695 } } peers { permanent_uuid: "9070a46ad55541b98e0e5e9b5507587b" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 39779 } } peers { permanent_uuid: "3997209206a04c878bfa0f605296e923" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 46691 } } }
I20260806 16:11:50.668685 7138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd6c4c688144ff5a4b37623a86b914d [sys.catalog]: This master's current role is: LEADER
W20260806 16:11:50.700944 7001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:50.695924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42126 (local address 127.8.49.61:39779)
0806 16:11:50.696095 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.696100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.699234 (+ 3134us) server_negotiation.cc:407] Connection header received
0806 16:11:50.699542 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.699548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.699623 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.699709 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.700735 (+ 1026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42126: BlockingRecv error: recv got EOF from 127.0.0.1:42126 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 16:11:50.704627 7092 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:50.696968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38938 (local address 127.8.49.60:46691)
0806 16:11:50.697112 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.697117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.701809 (+ 4692us) server_negotiation.cc:407] Connection header received
0806 16:11:50.703629 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.703634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.703712 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.703779 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.704426 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38938: BlockingRecv error: recv got EOF from 127.0.0.1:38938 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 16:11:50.705696 6897 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:50.698276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48674 (local address 127.8.49.62:42695)
0806 16:11:50.698482 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.698487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.702357 (+ 3870us) server_negotiation.cc:407] Connection header received
0806 16:11:50.702755 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.702760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.702869 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.702979 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.705511 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48674: BlockingRecv error: recv got EOF from 127.0.0.1:48674 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
W20260806 16:11:50.714193 7001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:50.710774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42138 (local address 127.8.49.61:39779)
0806 16:11:50.711023 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.711029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.711963 (+ 934us) server_negotiation.cc:407] Connection header received
0806 16:11:50.712684 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.712697 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.712856 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.712949 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.713995 (+ 1046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42138: BlockingRecv error: recv got EOF from 127.0.0.1:42138 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
W20260806 16:11:50.722611 7001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:50.720233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.8.49.61:39779)
0806 16:11:50.720401 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.720407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.721250 (+ 843us) server_negotiation.cc:407] Connection header received
0806 16:11:50.721519 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.721524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.721591 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.721673 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.722429 (+ 756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42148: BlockingRecv error: recv got EOF from 127.0.0.1:42148 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 16:11:50.734146 7001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:50.731665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42156 (local address 127.8.49.61:39779)
0806 16:11:50.731858 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.731864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.732777 (+ 913us) server_negotiation.cc:407] Connection header received
0806 16:11:50.733068 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.733073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.733144 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.733208 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.733975 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42156: BlockingRecv error: recv got EOF from 127.0.0.1:42156 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 16:11:50.744318 7001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:50.742093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42158 (local address 127.8.49.61:39779)
0806 16:11:50.742270 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.742275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.743069 (+ 794us) server_negotiation.cc:407] Connection header received
0806 16:11:50.743335 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.743340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.743405 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.743464 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.744141 (+ 677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42158: BlockingRecv error: recv got EOF from 127.0.0.1:42158 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 16:11:50.760648 7001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:50.757142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42164 (local address 127.8.49.61:39779)
0806 16:11:50.757337 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.757344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.758249 (+ 905us) server_negotiation.cc:407] Connection header received
0806 16:11:50.758593 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.758598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.758680 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.758768 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.760373 (+ 1605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42164: BlockingRecv error: recv got EOF from 127.0.0.1:42164 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 16:11:50.785485 7001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:50.782713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42174 (local address 127.8.49.61:39779)
0806 16:11:50.783032 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.783038 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.784032 (+ 994us) server_negotiation.cc:407] Connection header received
0806 16:11:50.784331 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.784336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.784417 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.784484 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.785277 (+ 793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42174: BlockingRecv error: recv got EOF from 127.0.0.1:42174 (error 108)
Metrics: {"server-negotiator.queue_time_us":188}
W20260806 16:11:50.838380 7001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:50.835779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42180 (local address 127.8.49.61:39779)
0806 16:11:50.835985 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.835992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.836956 (+ 964us) server_negotiation.cc:407] Connection header received
0806 16:11:50.837240 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.837244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.837317 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.837383 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.838158 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42180: BlockingRecv error: recv got EOF from 127.0.0.1:42180 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260806 16:11:50.908300 7001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:50.906395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.8.49.61:39779)
0806 16:11:50.906655 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:50.906663 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:50.906968 (+ 305us) server_negotiation.cc:407] Connection header received
0806 16:11:50.907222 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:50.907226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:50.907320 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:50.907422 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:50.908140 (+ 718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42186: BlockingRecv error: recv got EOF from 127.0.0.1:42186 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
W20260806 16:11:51.042815 7001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:51.040679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42194 (local address 127.8.49.61:39779)
0806 16:11:51.041018 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:51.041023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:51.041489 (+ 466us) server_negotiation.cc:407] Connection header received
0806 16:11:51.041709 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:51.041715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:51.041775 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:51.041886 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:51.042633 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42194: BlockingRecv error: recv got EOF from 127.0.0.1:42194 (error 108)
Metrics: {"server-negotiator.queue_time_us":190}
W20260806 16:11:51.305313 7001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:51.303097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42202 (local address 127.8.49.61:39779)
0806 16:11:51.303286 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:51.303292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:51.303923 (+ 631us) server_negotiation.cc:407] Connection header received
0806 16:11:51.304192 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:51.304195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:51.304314 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:51.304442 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:51.305110 (+ 668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42202: BlockingRecv error: recv got EOF from 127.0.0.1:42202 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:11:51.584961 7137 catalog_manager.cc:1259] Loaded cluster ID: 28deca68e7044400afc7efb4dcb7b69e
I20260806 16:11:51.585136 7137 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923: loading cluster ID for follower catalog manager: success
I20260806 16:11:51.587268 7137 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923: acquiring CA information for follower catalog manager: success
I20260806 16:11:51.588856 7137 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3997209206a04c878bfa0f605296e923: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:11:51.669502 7171 catalog_manager.cc:1259] Loaded cluster ID: 28deca68e7044400afc7efb4dcb7b69e
I20260806 16:11:51.669600 7171 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b: loading cluster ID for follower catalog manager: success
I20260806 16:11:51.671483 7171 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b: acquiring CA information for follower catalog manager: success
I20260806 16:11:51.672958 7171 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9070a46ad55541b98e0e5e9b5507587b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:11:51.827256 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:51.820360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42214 (local address 127.8.49.61:39779)
0806 16:11:51.820798 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:51.820802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:51.821060 (+ 258us) server_negotiation.cc:407] Connection header received
0806 16:11:51.821150 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:51.821153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:51.821310 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:51.821465 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:51.821862 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:51.822692 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:51.824691 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:51.825089 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:51.825215 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:51.825960 (+ 745us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:51.826489 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:51.826659 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:51.826939 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 16:11:51.827029 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":125,"threads_started":1}
I20260806 16:11:51.836037 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:51.831818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38946 (local address 127.8.49.60:46691)
0806 16:11:51.832204 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:51.832208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:51.832222 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:11:51.832320 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:51.832323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:51.832462 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:51.832606 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:51.832854 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:51.833414 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:51.834292 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:51.834530 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:51.834647 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:51.835042 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:51.835560 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:51.835710 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:51.835772 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 16:11:51.835838 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":125,"threads_started":1}
I20260806 16:11:51.846513 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:51.842656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38954 (local address 127.8.49.60:46691)
0806 16:11:51.842787 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:51.842792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:51.842807 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:11:51.842888 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:51.842890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:51.842940 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:51.843025 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:51.843406 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:51.844048 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:51.845302 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:51.845547 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:51.845677 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:51.845968 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:51.846061 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:51.846145 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:51.846276 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 16:11:51.846340 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 16:11:51.847420 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:51.842014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.8.49.61:39779)
0806 16:11:51.842202 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:51.842210 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:51.842427 (+ 217us) server_negotiation.cc:407] Connection header received
0806 16:11:51.842848 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:51.842854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:51.842929 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:51.842990 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:51.843518 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:51.844313 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:51.845733 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:51.846137 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:51.846236 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:51.846706 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:51.846886 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:51.846949 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:51.847179 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 16:11:51.847283 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 16:11:51.848835 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:51.841572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48684 (local address 127.8.49.62:42695)
0806 16:11:51.842091 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:51.842097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:51.842644 (+ 547us) server_negotiation.cc:407] Connection header received
0806 16:11:51.843055 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:51.843059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:51.843321 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:51.843474 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:51.843970 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:51.844835 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:51.846270 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:51.846701 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:51.846814 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:51.847408 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:51.847942 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:51.848048 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:51.848449 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 16:11:51.848545 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":109,"threads_started":1}
I20260806 16:11:51.855065 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 6789
I20260806 16:11:51.873698 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 6901
I20260806 16:11:51.887984 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 7003
2026-08-06T16:11:51Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3925 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:11:51 dist-test-slave-7z32 krb5kdc[7287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:11:51 dist-test-slave-7z32 krb5kdc[7287](info): set up 2 sockets
Aug 06 16:11:51 dist-test-slave-7z32 krb5kdc[7287](info): commencing operation
krb5kdc: starting...
W20260806 16:11:53.240221 8388 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.009s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/test-user.keytab.
Aug 06 16:11:53 dist-test-slave-7z32 krb5kdc[7287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:11:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:11:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.49.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
Entry for principal kudu/127.8.49.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab.
I20260806 16:11:53.428619 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:40633
--webserver_interface=127.8.49.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:40633,127.8.49.61:45857,127.8.49.60:32811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:11:53.593518 7306 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:11:53.594012 7306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:53.594041 7306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:53.594091 7306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:53.599813 7306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:53.599956 7306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:53.600001 7306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:53.600028 7306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:53.600063 7306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:53.607075 7306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:40633,127.8.49.61:45857,127.8.49.60:32811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.62.keytab
--principal=kudu/127.8.49.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.62:40633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.49.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:53.608690 7306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:53.610540 7306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:53.623486 7316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:53.623417 7314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:53.623711 7313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:53.623965 7306 server_base.cc:1034] running on GCE node
I20260806 16:11:53.624727 7306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:53.626158 7306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:53.629612 7306 hybrid_clock.cc:648] HybridClock initialized: now 1786032713629468 us; error 155 us; skew 500 ppm
Aug 06 16:11:53 dist-test-slave-7z32 krb5kdc[7287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:53.633690 7306 init.cc:377] Logged in from keytab as kudu/127.8.49.62@KRBTEST.COM (short username kudu)
I20260806 16:11:53.635174 7306 webserver.cc:460] Webserver started at http://127.8.49.62:34659/ using document root <none> and password file <none>
I20260806 16:11:53.635953 7306 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:53.636070 7306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:53.636358 7306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:53.639046 7306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/data/instance:
uuid: "b24c37fab7de4850bf3732dc11b80db4"
format_stamp: "Formatted at 2026-08-06 16:11:53 on dist-test-slave-7z32"
server_key: "488990d8cdb5e51af31ff362b2cfc6a4"
server_key_iv: "dc282cd5adab6bf70848b5325693f6b9"
server_key_version: "encryptionkey@0"
I20260806 16:11:53.639758 7306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/wal/instance:
uuid: "b24c37fab7de4850bf3732dc11b80db4"
format_stamp: "Formatted at 2026-08-06 16:11:53 on dist-test-slave-7z32"
server_key: "488990d8cdb5e51af31ff362b2cfc6a4"
server_key_iv: "dc282cd5adab6bf70848b5325693f6b9"
server_key_version: "encryptionkey@0"
I20260806 16:11:53.645296 7306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 16:11:53.649127 7327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:53.651399 7306 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:11:53.651648 7306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/wal
uuid: "b24c37fab7de4850bf3732dc11b80db4"
format_stamp: "Formatted at 2026-08-06 16:11:53 on dist-test-slave-7z32"
server_key: "488990d8cdb5e51af31ff362b2cfc6a4"
server_key_iv: "dc282cd5adab6bf70848b5325693f6b9"
server_key_version: "encryptionkey@0"
I20260806 16:11:53.651814 7306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:53.668354 7306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:53.672837 7306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:53.673115 7306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:53.683148 7306 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.62:40633
I20260806 16:11:53.683597 7400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.62:40633 every 8 connection(s)
I20260806 16:11:53.684818 7306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/data/info.pb
I20260806 16:11:53.689654 7401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:53.691922 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 7306
I20260806 16:11:53.692070 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-0/wal/instance
I20260806 16:11:53.692533 8388 external_mini_cluster.cc:1428] Setting key 62a3baf2e79fcf30d935d94898e5ec8e
I20260806 16:11:53.697332 7401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.49.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.61@KRBTEST.COM" created.
Aug 06 16:11:53 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:53.730336 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.703334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.8.49.62:40633)
0806 16:11:53.704823 (+ 1489us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:53.704847 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:53.705426 (+ 579us) server_negotiation.cc:407] Connection header received
0806 16:11:53.708015 (+ 2589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:53.708024 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:53.708325 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:53.708742 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:53.710385 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:53.712988 (+ 2603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:53.715255 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:53.716178 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:53.716498 (+ 320us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:53.721365 (+ 4867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:53.721372 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:53.721394 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:53.721422 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:53.724657 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:53.725652 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:53.725660 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:53.725661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:53.725816 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:53.726255 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:53.726259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:53.726261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:53.726807 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:53.727398 (+ 591us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:53.727913 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 16:11:53.728399 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":1283,"spinlock_wait_cycles":123776,"thread_start_us":801,"threads_started":1}
I20260806 16:11:53.730336 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.703334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:40633 (local address 127.0.0.1:39940)
0806 16:11:53.704864 (+ 1530us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:53.704967 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:53.707083 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:53.708834 (+ 1751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:53.708842 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:53.709534 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:53.709995 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:53.710018 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:53.713415 (+ 3397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:53.713421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:53.714793 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:53.714887 (+ 94us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:53.716547 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:53.716552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:53.716764 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:53.718228 (+ 1464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:53.718269 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:53.721119 (+ 2850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:53.724813 (+ 3694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:53.724829 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:53.724848 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:53.725127 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:53.725957 (+ 830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:53.725960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:53.725962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:53.726126 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:53.727359 (+ 1233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:53.727369 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:53.727593 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 16:11:53.727819 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 16:11:53.728600 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1104,"mutex_wait_us":288,"thread_start_us":689,"threads_started":1}
I20260806 16:11:53.734241 7401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:11:53.735816 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.735393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:45857 (local address 127.0.0.1:49574)
0806 16:11:53.735584 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:53.735707 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 16:11:53.736167 7328 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
I20260806 16:11:53.737829 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.737548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:45857 (local address 127.0.0.1:49578)
0806 16:11:53.737690 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:53.737752 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 16:11:53.738227 7401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:45857: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab.
Entry for principal kudu/127.8.49.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab.
I20260806 16:11:53.745069 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.61:45857
--webserver_interface=127.8.49.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab
--principal=kudu/127.8.49.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:40633,127.8.49.61:45857,127.8.49.60:32811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:11:53.787885 7401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } attempt: 1
I20260806 16:11:53.789479 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.788842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:45857 (local address 127.0.0.1:49592)
0806 16:11:53.789230 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:53.789344 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216}
I20260806 16:11:53.790863 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.790502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:45857 (local address 127.0.0.1:49600)
0806 16:11:53.790654 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:53.790737 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 16:11:53.791113 7401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:45857: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
I20260806 16:11:53.859371 7401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } attempt: 2
I20260806 16:11:53.860684 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.860198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:45857 (local address 127.0.0.1:49604)
0806 16:11:53.860456 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:53.860557 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 16:11:53.861948 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.861621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:45857 (local address 127.0.0.1:49608)
0806 16:11:53.861783 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:53.861864 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 16:11:53.862243 7401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:45857: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
W20260806 16:11:53.897022 7409 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:11:53.897415 7409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:53.897444 7409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:53.897492 7409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:53.902550 7409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:53.902664 7409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:53.902685 7409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:53.902704 7409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:53.902719 7409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:53.908797 7409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:40633,127.8.49.61:45857,127.8.49.60:32811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.61.keytab
--principal=kudu/127.8.49.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.61:45857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.49.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:53.910147 7409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:53.911381 7409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:53.923496 7417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:53.923488 7416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:53.923570 7409 server_base.cc:1034] running on GCE node
W20260806 16:11:53.923488 7420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:53.924742 7409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:53.926388 7409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:53.929847 7409 hybrid_clock.cc:648] HybridClock initialized: now 1786032713929797 us; error 63 us; skew 500 ppm
Aug 06 16:11:53 dist-test-slave-7z32 krb5kdc[7287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:53.933676 7409 init.cc:377] Logged in from keytab as kudu/127.8.49.61@KRBTEST.COM (short username kudu)
I20260806 16:11:53.934959 7409 webserver.cc:460] Webserver started at http://127.8.49.61:37421/ using document root <none> and password file <none>
I20260806 16:11:53.935650 7409 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:53.935757 7409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:53.936023 7409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:53.938414 7409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/data/instance:
uuid: "447c073b99ca4619b001fe591ddc2f0f"
format_stamp: "Formatted at 2026-08-06 16:11:53 on dist-test-slave-7z32"
server_key: "f33f2095756b577f09ce49e787de8ee9"
server_key_iv: "147c68b21867ba5f7856c9dfcb9549c3"
server_key_version: "encryptionkey@0"
I20260806 16:11:53.939169 7409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/wal/instance:
uuid: "447c073b99ca4619b001fe591ddc2f0f"
format_stamp: "Formatted at 2026-08-06 16:11:53 on dist-test-slave-7z32"
server_key: "f33f2095756b577f09ce49e787de8ee9"
server_key_iv: "147c68b21867ba5f7856c9dfcb9549c3"
server_key_version: "encryptionkey@0"
I20260806 16:11:53.944303 7409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 16:11:53.947903 7429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:53.949720 7409 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:11:53.949959 7409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/wal
uuid: "447c073b99ca4619b001fe591ddc2f0f"
format_stamp: "Formatted at 2026-08-06 16:11:53 on dist-test-slave-7z32"
server_key: "f33f2095756b577f09ce49e787de8ee9"
server_key_iv: "147c68b21867ba5f7856c9dfcb9549c3"
server_key_version: "encryptionkey@0"
I20260806 16:11:53.950163 7409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:53.953519 7401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } attempt: 3
I20260806 16:11:53.954792 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.954419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:45857 (local address 127.0.0.1:49620)
0806 16:11:53.954607 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:53.954694 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 16:11:53.956002 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.955574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:45857 (local address 127.0.0.1:49624)
0806 16:11:53.955785 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:53.955881 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 16:11:53.956236 7401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.61:45857: Network error: Client connection negotiation failed: client connection to 127.8.49.61:45857: connect: Connection refused (error 111)
I20260806 16:11:53.967310 7409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:53.971827 7409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:53.972054 7409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:53.981365 7409 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.61:45857
I20260806 16:11:53.981947 7504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.61:45857 every 8 connection(s)
I20260806 16:11:53.982604 7409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/data/info.pb
I20260806 16:11:53.987061 7505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:53.988617 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 7409
I20260806 16:11:53.988816 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-1/wal/instance
I20260806 16:11:53.989238 8388 external_mini_cluster.cc:1428] Setting key d9150abf5f417d5523e463cdadf4a4c3
I20260806 16:11:53.995219 7505 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.49.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.49.60@KRBTEST.COM" created.
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:54.022931 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.998576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.8.49.62:40633)
0806 16:11:53.998780 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:53.998785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.000854 (+ 2069us) server_negotiation.cc:407] Connection header received
0806 16:11:54.002279 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.002284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.002368 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.002505 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.005886 (+ 3381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.007024 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.008760 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.009047 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.009189 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.016352 (+ 7163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.016356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.016359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.016409 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.018146 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.019546 (+ 1400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.019550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.019552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.019724 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.020341 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.020348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.020352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.020681 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.020802 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.022473 (+ 1671us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.022676 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 16:11:54.023651 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:53.999301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:40633 (local address 127.0.0.1:39950)
0806 16:11:54.000514 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.000652 (+ 138us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:54.001967 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:54.003192 (+ 1225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:54.003352 (+ 160us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:54.004202 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:54.005495 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.005550 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.007264 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.007268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.008555 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.008574 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.009304 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.009310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.009543 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.010426 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:54.010450 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:54.015931 (+ 5481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:54.018485 (+ 2554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.018500 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.018538 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.019291 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.019917 (+ 626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.019920 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.019922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.020175 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.020857 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:54.020866 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:54.021309 (+ 443us) client_negotiation.cc:769] Sending connection context
0806 16:11:54.021554 (+ 245us) client_negotiation.cc:240] Negotiation successful
0806 16:11:54.022021 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":223,"threads_started":1}
I20260806 16:11:54.026053 7505 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.49.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab.
Entry for principal kudu/127.8.49.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab.
I20260806 16:11:54.040800 8388 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrledPc/build/debug/bin/kudu
/tmp/dist-test-taskrledPc/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.60:32811
--webserver_interface=127.8.49.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab
--principal=kudu/127.8.49.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.49.20:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.49.62:40633,127.8.49.61:45857,127.8.49.60:32811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:11:54.042660 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.027080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:45857 (local address 127.0.0.1:49626)
0806 16:11:54.027288 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.027307 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:54.027427 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:54.028659 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:54.028662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:54.028946 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:54.029244 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.029259 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.031124 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.031127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.031863 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.031875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.033081 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.033090 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.033279 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.034128 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:54.034176 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:54.036574 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:54.039550 (+ 2976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.039564 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.039566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.040036 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.040658 (+ 622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.040662 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.040664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.040812 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.041738 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:54.041745 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:54.041923 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 16:11:54.042066 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 16:11:54.042312 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 16:11:54.042670 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.027345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49626 (local address 127.8.49.61:45857)
0806 16:11:54.027946 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.027967 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.028000 (+ 33us) server_negotiation.cc:407] Connection header received
0806 16:11:54.028063 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.028069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.028440 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.028618 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.029502 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.030720 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.032224 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.032833 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.032962 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.036782 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.036800 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.036820 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.036846 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.039265 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.040300 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.040309 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.040311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.040446 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.041095 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.041101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.041105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.041592 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.041738 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.042238 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.042449 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":292,"threads_started":1}
I20260806 16:11:54.045576 7505 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:11:54.046941 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.046450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60566)
0806 16:11:54.046643 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.046777 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260806 16:11:54.047267 7436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
I20260806 16:11:54.048512 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.048222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60572)
0806 16:11:54.048368 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.048431 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 16:11:54.048754 7505 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:32811: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
I20260806 16:11:54.098142 7505 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } attempt: 1
I20260806 16:11:54.099340 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.099018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60580)
0806 16:11:54.099175 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.099255 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 16:11:54.099428 7401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } attempt: 4
I20260806 16:11:54.100419 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.100031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60582)
0806 16:11:54.100257 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.100315 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260806 16:11:54.100687 7505 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:32811: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
I20260806 16:11:54.111833 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.100365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.8.49.61:45857)
0806 16:11:54.100562 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.100567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.100592 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:11:54.100734 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.100741 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.100799 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.100899 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.101690 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.102297 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.103613 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.103927 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.104062 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.107741 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.107745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.107748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.107780 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.109596 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.110283 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.110287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.110288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.110373 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.110705 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.110710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.110718 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.111035 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.111162 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.111494 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.111630 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 16:11:54.111833 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.100168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:45857 (local address 127.0.0.1:49640)
0806 16:11:54.100444 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.100462 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:54.100580 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:54.100990 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:54.100995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:54.101278 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:54.101477 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.101485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.102466 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.102470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.103407 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.103420 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.104181 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.104185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.104340 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.105227 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:54.105255 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:54.107502 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:54.109792 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.109796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.109799 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.110148 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.110488 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.110492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.110494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.110596 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.111151 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:54.111155 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:54.111296 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 16:11:54.111418 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 16:11:54.111631 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
I20260806 16:11:54.113807 7401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:11:54.114969 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.114541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60594)
0806 16:11:54.114686 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.114747 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 16:11:54.116122 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.115807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60598)
0806 16:11:54.116007 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.116063 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260806 16:11:54.116436 7401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:32811: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
I20260806 16:11:54.168967 7505 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } attempt: 2
I20260806 16:11:54.170291 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.169830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60612)
0806 16:11:54.170100 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.170192 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 16:11:54.171482 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.171147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60620)
0806 16:11:54.171333 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.171413 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 16:11:54.171655 7505 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:32811: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
I20260806 16:11:54.175673 7401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } attempt: 1
I20260806 16:11:54.176906 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.176451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60634)
0806 16:11:54.176718 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.176801 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260806 16:11:54.178030 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.177741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60646)
0806 16:11:54.177862 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.177966 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 16:11:54.178206 7401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:32811: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
W20260806 16:11:54.212373 7513 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:11:54.212914 7513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:11:54.213001 7513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:11:54.213110 7513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:11:54.218228 7513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:11:54.218395 7513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:11:54.218436 7513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:11:54.218468 7513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:11:54.218506 7513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:11:54.225531 7513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.49.20:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.49.62:40633,127.8.49.61:45857,127.8.49.60:32811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/krb5kdc/kudu_127.8.49.60.keytab
--principal=kudu/127.8.49.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.49.60:32811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.49.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:19:10 UTC on ccycloud.quasar-onbjwb.root.comops.site
build id 4570623
I20260806 16:11:54.227401 7513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:11:54.229041 7513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:11:54.241523 7524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:54.241504 7521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:11:54.241545 7520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:11:54.242396 7513 server_base.cc:1034] running on GCE node
I20260806 16:11:54.243386 7513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:11:54.244853 7513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:11:54.245424 7401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } attempt: 2
I20260806 16:11:54.246402 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.246089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60660)
0806 16:11:54.246246 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.246314 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 16:11:54.247224 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.246994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60670)
0806 16:11:54.247112 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.247167 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 16:11:54.247479 7401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:32811: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
I20260806 16:11:54.248273 7513 hybrid_clock.cc:648] HybridClock initialized: now 1786032714248204 us; error 87 us; skew 500 ppm
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032714, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:11:54.252852 7513 init.cc:377] Logged in from keytab as kudu/127.8.49.60@KRBTEST.COM (short username kudu)
I20260806 16:11:54.254696 7513 webserver.cc:460] Webserver started at http://127.8.49.60:37191/ using document root <none> and password file <none>
I20260806 16:11:54.255540 7513 fs_manager.cc:359] Metadata directory not provided
I20260806 16:11:54.255623 7513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:11:54.255884 7513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:11:54.258560 7513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/data/instance:
uuid: "ac8fd4a84d414678b94397ab333b51bd"
format_stamp: "Formatted at 2026-08-06 16:11:54 on dist-test-slave-7z32"
server_key: "c676bb458965d941cdaf08eea4263041"
server_key_iv: "732d97236c2283aea95a98e4900020e1"
server_key_version: "encryptionkey@0"
I20260806 16:11:54.259353 7513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/wal/instance:
uuid: "ac8fd4a84d414678b94397ab333b51bd"
format_stamp: "Formatted at 2026-08-06 16:11:54 on dist-test-slave-7z32"
server_key: "c676bb458965d941cdaf08eea4263041"
server_key_iv: "732d97236c2283aea95a98e4900020e1"
server_key_version: "encryptionkey@0"
I20260806 16:11:54.262885 7505 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } attempt: 3
I20260806 16:11:54.264307 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.263869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60680)
0806 16:11:54.264124 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.264209 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 16:11:54.264957 7513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:11:54.265590 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.265232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60690)
0806 16:11:54.265446 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.265512 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260806 16:11:54.265893 7505 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.49.60:32811: Network error: Client connection negotiation failed: client connection to 127.8.49.60:32811: connect: Connection refused (error 111)
I20260806 16:11:54.268760 7532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:11:54.270895 7513 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 16:11:54.271124 7513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/data,/tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/wal
uuid: "ac8fd4a84d414678b94397ab333b51bd"
format_stamp: "Formatted at 2026-08-06 16:11:54 on dist-test-slave-7z32"
server_key: "c676bb458965d941cdaf08eea4263041"
server_key_iv: "732d97236c2283aea95a98e4900020e1"
server_key_version: "encryptionkey@0"
I20260806 16:11:54.271286 7513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:11:54.291333 7513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:11:54.295163 7513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:11:54.295414 7513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:11:54.304342 7513 rpc_server.cc:307] RPC server started. Bound to: 127.8.49.60:32811
I20260806 16:11:54.304903 7608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.49.60:32811 every 8 connection(s)
I20260806 16:11:54.305604 7513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/data/info.pb
I20260806 16:11:54.310017 7609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:11:54.314780 8388 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrledPc/build/debug/bin/kudu as pid 7513
I20260806 16:11:54.315076 8388 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrledPc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786032364843570-8388-0/minicluster-data/master-2/wal/instance
I20260806 16:11:54.315480 8388 external_mini_cluster.cc:1428] Setting key ec5c916fa34ff36be78522c48e0c1a6b
I20260806 16:11:54.315951 7609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032714, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
I20260806 16:11:54.333254 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.319514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39958 (local address 127.8.49.62:40633)
0806 16:11:54.319725 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.319733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.320634 (+ 901us) server_negotiation.cc:407] Connection header received
0806 16:11:54.320992 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.321000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.321085 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.321172 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.322965 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.323537 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.324983 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.325226 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.325347 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.328542 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.328545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.328547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.328579 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.330504 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.331253 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.331257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.331259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.331359 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.331788 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.331791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.331793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.332044 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.332173 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.332823 (+ 650us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.333070 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 16:11:54.334579 7613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.317822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.8.49.60:32811)
0806 16:11:54.318633 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.318653 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.318742 (+ 89us) server_negotiation.cc:407] Connection header received
0806 16:11:54.319978 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.320007 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.320467 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.320816 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.321752 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.322787 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.324220 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.325115 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.325356 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.328936 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.328945 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.328964 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.328996 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.331557 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.332199 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.332205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.332207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.332296 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.332614 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.332618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.332620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.332962 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.333228 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.333991 (+ 763us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.334172 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":145,"server-negotiator.queue_time_us":531,"spinlock_wait_cycles":1408,"thread_start_us":244,"threads_started":1}
I20260806 16:11:54.334583 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.319262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.62:40633 (local address 127.0.0.1:39958)
0806 16:11:54.320354 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.320444 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:54.320763 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:54.321508 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:54.321519 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:54.322286 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:54.322678 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.322702 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.323722 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.323725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.324820 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.324833 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.325415 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.325420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.325617 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.326334 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:54.326352 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:54.328334 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:54.330659 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.330672 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.330687 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.331132 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.331471 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.331474 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.331477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.331669 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.332176 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:54.332187 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:54.332517 (+ 330us) client_negotiation.cc:769] Sending connection context
0806 16:11:54.332748 (+ 231us) client_negotiation.cc:240] Negotiation successful
0806 16:11:54.333296 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":94,"thread_start_us":193,"threads_started":1}
I20260806 16:11:54.337109 7609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032714, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.62@KRBTEST.COM
I20260806 16:11:54.347791 7401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } attempt: 3
I20260806 16:11:54.351567 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.338308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.8.49.61:45857)
0806 16:11:54.338531 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.338537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.338794 (+ 257us) server_negotiation.cc:407] Connection header received
0806 16:11:54.339023 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.339027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.339092 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.339198 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.340108 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.341004 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.342701 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.343138 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.343296 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.346651 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.346656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.346660 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.346711 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.349432 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.349938 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.349941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.349947 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.350043 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.350411 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.350415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.350417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.350720 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.350883 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.351272 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.351423 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 16:11:54.351563 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.337997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.61:45857 (local address 127.0.0.1:49656)
0806 16:11:54.338718 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.338730 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:54.338865 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:54.339246 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:54.339251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:54.339475 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:54.339655 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.339663 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.341219 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.341223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.342406 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.342427 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.343355 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.343360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.343487 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.344241 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:54.344278 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:54.346377 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:54.349576 (+ 3199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.349579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.349582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.349812 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.350148 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.350152 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.350154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.350245 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.350906 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:54.350909 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:54.351064 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 16:11:54.351198 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 16:11:54.351364 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642}
I20260806 16:11:54.353137 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.338367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39962 (local address 127.8.49.62:40633)
0806 16:11:54.338539 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.338544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.338563 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:11:54.338658 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.338661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.338758 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.338987 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.340111 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.340983 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.342665 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.343137 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.343296 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.347634 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.347640 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.347648 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.347696 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.350206 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.350735 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.350740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.350742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.350951 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.351405 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.351409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.351411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.351643 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.351931 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.352781 (+ 850us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.352942 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:11:54.353096 7609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.62@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
I20260806 16:11:54.360915 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.348399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60698)
0806 16:11:54.348691 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.348717 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:54.348855 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:54.349213 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:54.349218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:54.349713 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:54.350009 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.350021 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.351278 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.351282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.352091 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.352102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.352786 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.352793 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.352938 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.353515 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:54.353532 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:54.356186 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:54.358519 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.358524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.358526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.358911 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.359316 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.359320 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.359372 (+ 52us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.359551 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.360195 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:54.360200 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:54.360337 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 16:11:54.360465 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 16:11:54.360642 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260806 16:11:54.361465 7613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.348733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.8.49.60:32811)
0806 16:11:54.348916 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.348921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.348942 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:11:54.348992 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.348996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.349053 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.349157 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.350231 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.351132 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.352256 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.352615 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.352746 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.356346 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.356359 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.356362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.356386 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.358368 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.359029 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.359033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.359035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.359149 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.359686 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.359692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.359697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.359969 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.360109 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.361235 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.361349 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032714, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.60@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.49.61@KRBTEST.COM
I20260806 16:11:54.365975 7401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4: Bootstrap starting.
I20260806 16:11:54.366432 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.353799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60710)
0806 16:11:54.354380 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.354397 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:54.354489 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:54.355920 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:54.355923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:54.356116 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:54.356237 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.356244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.357959 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.357962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.358705 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.358715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.359514 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.359518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.359611 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.360215 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:54.360228 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:54.362120 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:54.364488 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.364491 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.364493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.364724 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.365039 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.365042 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.365044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.365129 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.365822 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:54.365826 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:54.365973 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 16:11:54.366112 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 16:11:54.366277 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499}
I20260806 16:11:54.367023 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.354582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.8.49.60:32811)
0806 16:11:54.355407 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.355424 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.355443 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:11:54.355492 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.355496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.355554 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.355688 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.356358 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.357846 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.358900 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.359255 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.359363 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.362293 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.362297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.362300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.362326 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.364350 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.364836 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.364842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.364844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.364938 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.365229 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.365234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.365236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.365654 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.365768 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.366682 (+ 914us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.366813 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":240,"threads_started":1}
I20260806 16:11:54.367990 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.354931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49668 (local address 127.8.49.61:45857)
0806 16:11:54.355145 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.355149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.355162 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:11:54.355206 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.355208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.355248 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.355356 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.356227 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.357130 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.358290 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.358689 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.358829 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.363117 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.363121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.363124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.363161 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.365236 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.365853 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.365857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.365858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.365957 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.366272 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.366275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.366277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.366521 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.366875 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.367723 (+ 848us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.367849 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 16:11:54.369861 7609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd: Bootstrap starting.
I20260806 16:11:54.372833 7609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:54.373870 7609 log.cc:824] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:54.374372 7401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:54.375756 7401 log.cc:824] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:54.376334 8388 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:11:54.377252 7609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd: No bootstrap required, opened a new log
I20260806 16:11:54.378621 7401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4: No bootstrap required, opened a new log
I20260806 16:11:54.378206 7368 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b24c37fab7de4850bf3732dc11b80db4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39962
I20260806 16:11:54.381124 7609 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } }
I20260806 16:11:54.381453 7609 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac8fd4a84d414678b94397ab333b51bd, State: Initialized, Role: FOLLOWER
I20260806 16:11:54.381805 7609 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } }
I20260806 16:11:54.382392 7621 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } } }
I20260806 16:11:54.382557 7621 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:54.382581 7401 raft_consensus.cc:348] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } }
I20260806 16:11:54.383066 7401 raft_consensus.cc:729] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24c37fab7de4850bf3732dc11b80db4, State: Initialized, Role: FOLLOWER
I20260806 16:11:54.383484 7609 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:54.383478 7401 consensus_queue.cc:260] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } }
I20260806 16:11:54.384207 7622 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } } }
I20260806 16:11:54.384429 7622 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:54.385340 7401 sys_catalog.cc:564] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:54.385402 7368 raft_consensus.cc:482] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 16:11:54.385519 7368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:54.387502 7368 raft_consensus.cc:504] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } }
I20260806 16:11:54.389693 7368 leader_election.cc:288] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [CANDIDATE]: Term 1 election: Requested vote from peers 447c073b99ca4619b001fe591ddc2f0f (127.8.49.61:45857), ac8fd4a84d414678b94397ab333b51bd (127.8.49.60:32811)
W20260806 16:11:54.391619 7639 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:11:54.391858 7639 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:54.388599 7513 master_runner.cc:186] Error getting master registration for 127.8.49.62:40633: OK, instance_id { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" instance_seqno: 1786032713680498 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:11:54.392620 7513 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:11:54.390985 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b24c37fab7de4850bf3732dc11b80db4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ac8fd4a84d414678b94397ab333b51bd"
I20260806 16:11:54.394793 7569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:54.392751 7306 master_runner.cc:186] Error getting master registration for 127.8.49.61:45857: OK, instance_id { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" instance_seqno: 1786032713978750 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:11:54.396710 7306 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:11:54.396998 7569 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24c37fab7de4850bf3732dc11b80db4 in term 1.
I20260806 16:11:54.397826 7332 leader_election.cc:302] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac8fd4a84d414678b94397ab333b51bd, b24c37fab7de4850bf3732dc11b80db4; no voters:
I20260806 16:11:54.398545 7622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:11:54.398949 7622 raft_consensus.cc:686] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [term 1 LEADER]: Becoming Leader. State: Replica: b24c37fab7de4850bf3732dc11b80db4, State: Running, Role: LEADER
I20260806 16:11:54.399443 7622 consensus_queue.cc:237] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } }
I20260806 16:11:54.401887 7622 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b24c37fab7de4850bf3732dc11b80db4. Latest consensus state: current_term: 1 leader_uuid: "b24c37fab7de4850bf3732dc11b80db4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } } }
I20260806 16:11:54.402285 7622 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [sys.catalog]: This master's current role is: LEADER
W20260806 16:11:54.407729 7654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:11:54.407871 7654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:54.409060 7505 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } attempt: 4
I20260806 16:11:54.413410 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.396936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49678 (local address 127.8.49.61:45857)
0806 16:11:54.397105 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.397110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.397357 (+ 247us) server_negotiation.cc:407] Connection header received
0806 16:11:54.398569 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.398580 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.398660 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.398780 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.403984 (+ 5204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.404565 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.405734 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.406052 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.406142 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.408578 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.408584 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.408589 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.408626 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.410670 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.411692 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.411699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.411704 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.411839 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.412209 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.412214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.412216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.412493 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.412653 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.413052 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.413200 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 16:11:54.414529 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.398069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60724 (local address 127.8.49.60:32811)
0806 16:11:54.398236 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.398241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.399724 (+ 1483us) server_negotiation.cc:407] Connection header received
0806 16:11:54.399890 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.399894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.399972 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.400090 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.401223 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.402815 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.404278 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.404838 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.404995 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.407551 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.407557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.407560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.407602 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.411706 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.412331 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.412335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.412337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.412449 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.412819 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.412824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.412828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.413115 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.413309 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.414148 (+ 839us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.414309 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:11:54.417770 7671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 16:11:54 dist-test-slave-7z32 krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032713, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.49.61@KRBTEST.COM for kudu/127.8.49.60@KRBTEST.COM
I20260806 16:11:54.419984 7671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:11:54.425290 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.412435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.49.60:32811 (local address 127.0.0.1:60726)
0806 16:11:54.412768 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 16:11:54.412783 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:11:54.412876 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:11:54.413601 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:11:54.413604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:11:54.414000 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:11:54.414271 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.414282 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.415806 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.415811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.416991 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:11:54.417001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.417633 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.417636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:11:54.417713 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.418451 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:11:54.418469 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:11:54.420637 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:11:54.423172 (+ 2535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.423176 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.423179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.423535 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.423884 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:11:54.423887 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:11:54.423889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:11:54.424003 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:11:54.424648 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:11:54.424652 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:11:54.424789 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 16:11:54.424915 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 16:11:54.425090 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242}
W20260806 16:11:54.426021 7336 tablet.cc:2369] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:11:54.427199 7613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.413047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60726 (local address 127.8.49.60:32811)
0806 16:11:54.413192 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.413196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.413209 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:11:54.413254 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.413257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.413306 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.413384 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.414729 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.415629 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.417149 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.417479 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.417632 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.420973 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.420978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.420981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.421020 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.422967 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.423669 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.423673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.423675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.423772 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.424152 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.424161 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.424164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.424495 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.424720 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.426792 (+ 2072us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.427006 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:11:54.431905 7505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f: Bootstrap starting.
I20260806 16:11:54.432577 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:54.399248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.8.49.62:40633)
0806 16:11:54.399381 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.399386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.408393 (+ 9007us) server_negotiation.cc:407] Connection header received
0806 16:11:54.408664 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.408670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.408742 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.408822 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.410017 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.410967 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.415130 (+ 4163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:54.415558 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:54.415684 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:54.420476 (+ 4792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:11:54.420481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:11:54.420485 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:11:54.420522 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:11:54.422944 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.423647 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.423651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.423653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.423762 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:11:54.430868 (+ 7106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:11:54.430874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:11:54.430876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:11:54.431252 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:11:54.431438 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:54.431816 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 16:11:54.432239 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:11:54.435307 7505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f: Neither blocks nor log segments found. Creating new log.
I20260806 16:11:54.436362 7505 log.cc:824] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f: Log is configured to *not* fsync() on all Append() calls
I20260806 16:11:54.445138 7569 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [term 1 FOLLOWER]: Refusing update from remote peer b24c37fab7de4850bf3732dc11b80db4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:54.445817 7505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f: No bootstrap required, opened a new log
I20260806 16:11:54.446415 7666 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:54.450685 7505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } }
I20260806 16:11:54.451323 7505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447c073b99ca4619b001fe591ddc2f0f, State: Initialized, Role: FOLLOWER
I20260806 16:11:54.451794 7505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } }
I20260806 16:11:54.454649 7505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:11:54.455058 7463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:11:54.460606 7621 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b24c37fab7de4850bf3732dc11b80db4. Latest consensus state: current_term: 1 leader_uuid: "b24c37fab7de4850bf3732dc11b80db4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } } }
I20260806 16:11:54.460862 7621 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:54.463636 7666 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b24c37fab7de4850bf3732dc11b80db4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } } }
I20260806 16:11:54.463842 7666 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [sys.catalog]: This master's current role is: LEADER
I20260806 16:11:54.464004 7673 mvcc.cc:205] Tried to move back new op lower bound from 7315589998342078464 to 7315589998184804352. Current Snapshot: MvccSnapshot[applied={T|T < 7315589998342078464}]
I20260806 16:11:54.465634 7463 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [term 1 FOLLOWER]: Refusing update from remote peer b24c37fab7de4850bf3732dc11b80db4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:11:54.466738 7622 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:11:54.467059 7676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } } }
I20260806 16:11:54.467236 7676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:54.462765 7467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b24c37fab7de4850bf3732dc11b80db4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "447c073b99ca4619b001fe591ddc2f0f"
I20260806 16:11:54.467681 7621 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b24c37fab7de4850bf3732dc11b80db4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } } }
I20260806 16:11:54.467828 7621 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:54.469174 7678 mvcc.cc:205] Tried to move back new op lower bound from 7315589998342078464 to 7315589998184804352. Current Snapshot: MvccSnapshot[applied={T|T < 7315589998342078464}]
I20260806 16:11:54.470289 7467 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24c37fab7de4850bf3732dc11b80db4 in term 1.
W20260806 16:11:54.475404 7440 tablet.cc:2369] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:11:54.477988 7671 catalog_manager.cc:1347] Generated new cluster ID: e0825b897fc346cea5b04a59abf9aff6
I20260806 16:11:54.478106 7671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:11:54.478479 7676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b24c37fab7de4850bf3732dc11b80db4. Latest consensus state: current_term: 1 leader_uuid: "b24c37fab7de4850bf3732dc11b80db4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } } }
I20260806 16:11:54.478652 7676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:54.478775 7676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b24c37fab7de4850bf3732dc11b80db4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } } }
I20260806 16:11:54.478884 7676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:11:54.485569 7684 mvcc.cc:205] Tried to move back new op lower bound from 7315589998342078464 to 7315589998184804352. Current Snapshot: MvccSnapshot[applied={T|T < 7315589998342078464}]
I20260806 16:11:54.498194 7666 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b24c37fab7de4850bf3732dc11b80db4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24c37fab7de4850bf3732dc11b80db4" member_type: VOTER last_known_addr { host: "127.8.49.62" port: 40633 } } peers { permanent_uuid: "447c073b99ca4619b001fe591ddc2f0f" member_type: VOTER last_known_addr { host: "127.8.49.61" port: 45857 } } peers { permanent_uuid: "ac8fd4a84d414678b94397ab333b51bd" member_type: VOTER last_known_addr { host: "127.8.49.60" port: 32811 } } }
I20260806 16:11:54.498435 7666 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4 [sys.catalog]: This master's current role is: LEADER
W20260806 16:11:54.503549 7699 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:11:54.503705 7699 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:11:54.505090 7671 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:11:54.507938 7671 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:11:54.538250 7671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b24c37fab7de4850bf3732dc11b80db4: Generated new TSK 0
I20260806 16:11:54.539819 7671 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:11:54.599205 7406 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.596195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39990 (local address 127.8.49.62:40633)
0806 16:11:54.596433 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.596439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.597404 (+ 965us) server_negotiation.cc:407] Connection header received
0806 16:11:54.597695 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.597700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.597770 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.597837 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.598996 (+ 1159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39990: BlockingRecv error: recv got EOF from 127.0.0.1:39990 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260806 16:11:54.602901 7512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.601366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49682 (local address 127.8.49.61:45857)
0806 16:11:54.601611 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.601617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.601640 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:11:54.601707 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.601711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.601774 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.601909 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.602667 (+ 758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49682: BlockingRecv error: recv got EOF from 127.0.0.1:49682 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 16:11:54.605726 7613 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.601902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60742 (local address 127.8.49.60:32811)
0806 16:11:54.602123 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.602128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.604336 (+ 2208us) server_negotiation.cc:407] Connection header received
0806 16:11:54.604570 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.604574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.604651 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.604715 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.605501 (+ 786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60742: BlockingRecv error: recv got EOF from 127.0.0.1:60742 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260806 16:11:54.613600 7512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.610336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.8.49.61:45857)
0806 16:11:54.610502 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.610508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.611642 (+ 1134us) server_negotiation.cc:407] Connection header received
0806 16:11:54.612307 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.612313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.612453 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.612541 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.613420 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49692: BlockingRecv error: recv got EOF from 127.0.0.1:49692 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 16:11:54.621722 7512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.619233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49694 (local address 127.8.49.61:45857)
0806 16:11:54.619408 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.619414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.620240 (+ 826us) server_negotiation.cc:407] Connection header received
0806 16:11:54.620563 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.620568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.620646 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.620719 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.621543 (+ 824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49694: BlockingRecv error: recv got EOF from 127.0.0.1:49694 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 16:11:54.632316 7512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.628581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49704 (local address 127.8.49.61:45857)
0806 16:11:54.628753 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.628758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.630043 (+ 1285us) server_negotiation.cc:407] Connection header received
0806 16:11:54.630514 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.630518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.630651 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.630757 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.632156 (+ 1399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49704: BlockingRecv error: recv got EOF from 127.0.0.1:49704 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 16:11:54.644510 7512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.641538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49718 (local address 127.8.49.61:45857)
0806 16:11:54.641752 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.641759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.642549 (+ 790us) server_negotiation.cc:407] Connection header received
0806 16:11:54.642886 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.642891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.642969 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.643037 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.644325 (+ 1288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49718: BlockingRecv error: recv got EOF from 127.0.0.1:49718 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260806 16:11:54.659672 7512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.657005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49730 (local address 127.8.49.61:45857)
0806 16:11:54.657230 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.657238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.658111 (+ 873us) server_negotiation.cc:407] Connection header received
0806 16:11:54.658437 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.658441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.658532 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.658621 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.659488 (+ 867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49730: BlockingRecv error: recv got EOF from 127.0.0.1:49730 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 16:11:54.681412 7512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.679750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49744 (local address 127.8.49.61:45857)
0806 16:11:54.679935 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.679942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.680273 (+ 331us) server_negotiation.cc:407] Connection header received
0806 16:11:54.680490 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.680494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.680570 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.680682 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.681248 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49744: BlockingRecv error: recv got EOF from 127.0.0.1:49744 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 16:11:54.718873 7512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.717037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49748 (local address 127.8.49.61:45857)
0806 16:11:54.717239 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.717244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.717448 (+ 204us) server_negotiation.cc:407] Connection header received
0806 16:11:54.717554 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.717558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.717634 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.717783 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.718687 (+ 904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49748: BlockingRecv error: recv got EOF from 127.0.0.1:49748 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260806 16:11:54.788640 7512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.786575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49750 (local address 127.8.49.61:45857)
0806 16:11:54.786803 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.786811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.787395 (+ 584us) server_negotiation.cc:407] Connection header received
0806 16:11:54.787664 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.787668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.787743 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.787847 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.788489 (+ 642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49750: BlockingRecv error: recv got EOF from 127.0.0.1:49750 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260806 16:11:54.923023 7512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:54.920820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49758 (local address 127.8.49.61:45857)
0806 16:11:54.921057 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:54.921063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:54.921428 (+ 365us) server_negotiation.cc:407] Connection header received
0806 16:11:54.921543 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:54.921547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:54.921658 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:54.921775 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:54.922603 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49758: BlockingRecv error: recv got EOF from 127.0.0.1:49758 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260806 16:11:55.183687 7512 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:11:55.182153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.8.49.61:45857)
0806 16:11:55.182408 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:55.182414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:55.182577 (+ 163us) server_negotiation.cc:407] Connection header received
0806 16:11:55.182782 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:55.182786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:55.182872 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:55.182975 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:11:55.183544 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49762: BlockingRecv error: recv got EOF from 127.0.0.1:49762 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
I20260806 16:11:55.393800 7639 catalog_manager.cc:1259] Loaded cluster ID: e0825b897fc346cea5b04a59abf9aff6
I20260806 16:11:55.393967 7639 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd: loading cluster ID for follower catalog manager: success
I20260806 16:11:55.396937 7639 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd: acquiring CA information for follower catalog manager: success
I20260806 16:11:55.399168 7639 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ac8fd4a84d414678b94397ab333b51bd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:11:55.505415 7699 catalog_manager.cc:1259] Loaded cluster ID: e0825b897fc346cea5b04a59abf9aff6
I20260806 16:11:55.505520 7699 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f: loading cluster ID for follower catalog manager: success
I20260806 16:11:55.507416 7699 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f: acquiring CA information for follower catalog manager: success
I20260806 16:11:55.508680 7699 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 447c073b99ca4619b001fe591ddc2f0f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:11:55.706262 7790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:55.700190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49764 (local address 127.8.49.61:45857)
0806 16:11:55.700764 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:55.700768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:55.700791 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:11:55.700859 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:55.700862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:55.701022 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:55.701178 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:55.701852 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:55.702488 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:55.703802 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:55.704124 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:55.704248 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:55.704834 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:55.705427 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:55.705629 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:55.705863 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 16:11:55.706032 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":140,"threads_started":1}
I20260806 16:11:55.719656 7798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:55.712861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.8.49.60:32811)
0806 16:11:55.713484 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:55.713489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:55.713503 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:11:55.713688 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:55.713691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:55.713847 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:55.713976 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:55.714501 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:55.715192 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:55.716773 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:55.717074 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:55.717200 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:55.717773 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:55.718789 (+ 1016us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:55.719123 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:55.719340 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 16:11:55.719434 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":441,"threads_started":1}
I20260806 16:11:55.731003 7798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:55.725127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60762 (local address 127.8.49.60:32811)
0806 16:11:55.725274 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:55.725279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:55.726331 (+ 1052us) server_negotiation.cc:407] Connection header received
0806 16:11:55.726725 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:55.726730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:55.726814 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:55.726912 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:55.727633 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:55.728245 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:55.729444 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:55.729697 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:55.729833 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:55.730152 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:55.730272 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:55.730359 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:55.730585 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 16:11:55.730719 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:11:55.733413 7790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:55.725075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49780 (local address 127.8.49.61:45857)
0806 16:11:55.725275 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:55.725281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:55.725913 (+ 632us) server_negotiation.cc:407] Connection header received
0806 16:11:55.726867 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:55.726872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:55.726950 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:55.727048 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:55.727468 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:55.729840 (+ 2372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:55.731961 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:55.732354 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:55.732491 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:55.732815 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:55.732929 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:55.733021 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:55.733158 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 16:11:55.733232 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 16:11:55.734277 7806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:11:55.724967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40004 (local address 127.8.49.62:40633)
0806 16:11:55.725993 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0806 16:11:55.726014 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:11:55.726039 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:11:55.726786 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:11:55.726790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:11:55.727038 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:11:55.727192 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:11:55.727848 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:55.730189 (+ 2341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:55.731781 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:11:55.732157 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:11:55.732333 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:11:55.732898 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:11:55.733378 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:11:55.733504 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 16:11:55.733973 (+ 469us) server_negotiation.cc:299] Negotiation successful
0806 16:11:55.734070 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":920,"thread_start_us":123,"threads_started":1}
I20260806 16:11:55.741451 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 7306
I20260806 16:11:55.767087 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 7409
I20260806 16:11:55.782797 8388 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrledPc/build/debug/bin/kudu with pid 7513
2026-08-06T16:11:55Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3899 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (7825 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (350929 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST